Re: Geode Resource Manager and newer Garbage Collectors

["evaristo.camarero@yahoo.es" <ev...@yahoo.es>]

 
Hi there,

 I completely agree that best choice is jumping to ZGC/ShenandoahGC.

 I have been doing some tests with both using OpenJDK 15.0.2.In general pause times are quite predictable even without tuning. Both GCs arenot tenured, and are able to make full GCs but mainly all the job is concurrentwith the app and pauses are quite ok (typically < 5 msecs)

 Regarding Geode Resource Manager when using heap-basedeviction, I think both GCs can use a similar approach:
 -  For ZGC '-XX:SoftMaxHeapSize’ could be a limitfor Geode to figure out when eviction should be triggered. If not set, in mytests ZGC tried to use the max heap size to the limit, taking benefit that cancollect garbage concurrently. In Java 16 it is even able to do almost all thework concurrently with expected pauses under 1 msec. (Check this blog from PerLiden ZGC What’s new in JDK 16, principal developerfor ZGC)
 

|  
|  
|  |  
ZGC What’s new in JDK 16
 
Per Liden
 
JDK 16 is out, and as usual, each new release comes with a bunch of new features, enhancements and bug fixes. ZG...
  |

  |

  |





I was testing with XX:SoftMaxHeapSizeand the soft limit was honored by the JVM

In general ZGC worked fine, when GC is triggeredyou can see a CPU spike that is negictible, but almost of the job is doneconcurrently with Geode




- Shenandoah has a differentparameter, -XX:ShenandoahMinFreeThreshold that can do the job. By defaultis 10%, so that means that Shenandoah GC is triggered when only 10% of the heapsize is free. Taking benefit of this, Geode Resource Manager heap-basedeviction could assume that if free heap is over the value according to theparameter during a window, probably eviction should kick-in.

I was testing it also, and worked fine and theJVM is honoring it properly


 


I think that implementation in both cases could be quitesimilar and I think could be useful to go ahead with both options and discardG1, considering that:

- -XX:SoftMaxHeapSize supportfor G1 looks stack for quite a long time so far

- Shenandoah has been backported to OpenJDK 11 (LTS).Probably for the people that prefers to use Java LTS, this is the best optionso far. One CON (a big one) is that Oracle builds are not including Shenandoah,but many other builds are available (including commercial support)
- ZGC is production ready in OpenJDK 15 and improved in Java16 to be a serious contender to Azul C4. Probably when next Java LTS is there(Java 17?), I think it will be the perfect match for Geode in my opinion

Do you think that heap based eviction is the main aspect to consider to make a proposal? Any other aspect that the proposal should include?

Thanks in advance,
/Cheva
 

 

 

    En miércoles, 31 de marzo de 2021 19:11:08 CEST, Anthony Baker <ba...@vmware.com> escribió:  
 
 In our testing we’ve found mixed results with G1 and Geode for many applications.  

- G1 does require significant tuning to achieve desired pause times on many workloads.  This is somewhat disappointing as in practice it requires a similar degree of effort/complexity as CMS.

- G1 tends to be at odds with the Geode Resource Manager when using heap-based eviction (this is a very common safety mechanism).  There may be some hope if '-XX:SoftMaxHeapSize’ is ever merged to relevant versions like jdk8.

I suspect the best choice is to skip G1 and go directly to Z / Shenandoah which are more targeted at low latency high memory workloads.  I would LOVE to see an RFC proposing this effort :-)


Anthony


> On Mar 31, 2021, at 6:58 AM, Jacob Barrett <ja...@vmware.com> wrote:
> 
> There has been lots of talk over the years but no action I am aware of. I would suggest we start with Z since it is more widely available. In benchmarking it has shown impressive out of the box improvements over heavily tuned CMS. Shenandoah is very similar in behavior and structure so picking it up after should be easy.
> 
>> On Mar 31, 2021, at 5:58 AM, Alberto Gomez <al...@est.tech> wrote:
>> 
>> 
>> Hi,
>> 
>> The Geode documentation on the Resource Manager ([1]) provides information about its integration with CMS and G1 although it seems to suggest that only CMS is well supported with it, or at least, it is the preferred alternative.
>> 
>> Given that CMS has been deprecated in JDK 9 (and will not be available in JDK 14) and that there are newer and more efficient GCs like Shenandoah and ZGC available (Shenandoah is supported in OpenJDK 11.0.9), I would like to know if there is any work or discussions ongoing in the community about the integration of ZGC and/or Shenandoah with Geode's Resource Manager.
>> 
>> Thanks in advance,
>> 
>> Alberto G.
>> 
>> [1]: https://geode.apache.org/docs/guide/113/managing/heap_use/heap_management.html

  

Re: Geode Resource Manager and newer Garbage Collectors

[Anthony Baker <ba...@vmware.com>]

Nice results!  Beyond integrating the ResourceManager heap-based eviction I would look into:

1) Do the GC statistics we gather make sense under Z / Shenandoah?
2) How should we modify gfsh to switch the default launch options (start server/locator) from CMS to something better?
3) What do we do about the 75% of the Java market still running JDK8?  Should we only switch the default collector when running with a newer JDK…?

Just IMO,
Anthony


On Apr 2, 2021, at 9:21 AM, evaristo.camarero@yahoo.es<ma...@yahoo.es> wrote:

Do you think that heap based eviction is the main aspect to consider to make a proposal? Any other aspect that the proposal should include?