[jira] [Commented] (FLINK-11396) MetricStore creates significant GC pressure

["Oleksandr Nitavskyi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746997#comment-16746997 ] 

Oleksandr Nitavskyi commented on FLINK-11396:
---------------------------------------------

On GC tuning side, G1 ergonomic concluded we need to have a small Young Memory Pool (because short living object are created only during UI usage, which is rare), so alternative solution would be to set Yound Memory Pool size via parameters or use ParallelGC.

> MetricStore creates significant GC pressure
> -------------------------------------------
>
>                 Key: FLINK-11396
>                 URL: https://issues.apache.org/jira/browse/FLINK-11396
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Oleksandr Nitavskyi
>            Priority: Major
>         Attachments: GC_parallel_flink_job_manager.log, g1.txt, gc_example.txt
>
>
> On Flink version 1.6.x we observe a significant increase of the latency in UI. 
> After performance profiling, we have concluded that during UI rendering back-end spends 50% of the time on GC pauses, which means that Flink 1.6.x is not friendly with G1 GC ergonomics configurations. 
> On Flink side, MetricStore creates a huge amount of short living objects for our job, which provokes a lot of Young GC pauses for a non-small job, with around 50 operators and 120 parallelisms. 
> Samples of GC logs are in attach.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)