[jira] [Commented] (ACCUMULO-3770) Replace GarbageCollectionLogger with JVM provided GC stats

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490782#comment-15490782 ] 

Josh Elser commented on ACCUMULO-3770:
--------------------------------------

Was talking with Bitweasil in IRC yesterday/today about this. He expressed some confusion about the GC logging we had. After digging, he found that we were actually logging the cumulative time spent in GC, not the time of the last cycle as expected.

bq.Where do these reports go?  Are they rolled-over like the regular logs?

Typically, you'll get a {{gc.log\[\-date\]}} style file, but I believe this is also configurable.

bq. The timeliness feature is nice, but that can be done in a separate SimpleTimer callback.

Agreed. Timeliness check is very good and should be considered separate to GC logs.

bq. Yeah, rollover is nice. Also does what java prints out have timestamps?

Yes. Here are some examples from the logs that HDP creates for the secondary namenode:

{noformat}
2016-09-14T09:57:00.476+0000: 54036.340: [GC (Allocation Failure) 2016-09-14T09:57:00.476+0000: 54036.340: [ParNew: 106050K->1130K(118016K), 0.0680896 secs] 125622K->20712K(1035520K), 0.0683270 secs] [Times: user=0.13 sys=0.00, real=0.07 secs]
2016-09-14T12:12:44.975+0000: 62180.839: [GC (Allocation Failure) 2016-09-14T12:12:44.975+0000: 62180.840: [ParNew: 106090K->1157K(118016K), 0.0503738 secs] 125672K->20742K(1035520K), 0.0505503 secs] [Times: user=0.10 sys=0.00, real=0.05 secs]
2016-09-14T14:51:46.898+0000: 71722.762: [GC (Allocation Failure) 2016-09-14T14:51:46.898+0000: 71722.762: [ParNew: 106117K->697K(118016K), 0.0705879 secs] 125702K->20662K(1035520K), 0.0707647 secs] [Times: user=0.14 sys=0.00, real=0.07 secs]
{noformat}

Another perk is that we can use standard tools for parsing these logs. Here is a nice little article that explains how to read these lines https://plumbr.eu/blog/garbage-collection/understanding-garbage-collection-logs

> Replace GarbageCollectionLogger with JVM provided GC stats
> ----------------------------------------------------------
>
>                 Key: ACCUMULO-3770
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3770
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Josh Elser
>            Priority: Minor
>             Fix For: 2.0.0
>
>
> Hotspot provides some JVM opts that can be enabled which will print out status messages for garbage collection:
> {{-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -verbose:gc}}
> TabletServers have the {{GarbageCollectionLogger}} doing its own inspection of heap usage via {{Runtime}} and also tracking the real interval between calls to itself WRT the expected interval (to try to guess if there are issues getting resources to run code).
> As long as openjdk has support for the same options, I think removing the heap inspection would be good. Do we still want to preserve the "timeliness" check though? I'm not sure how valuable it is in real environments.
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)