You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 王铁 <37...@qq.com> on 2020/05/08 15:28:06 UTC
I have a wired issue

Hi there


I have a cluster with 5 brokers, they have same hardware(40Core, 128G MEM, 36T DISK), &nbsp;JVM version(1.8.0_144) and start parameters&nbsp;


there is one broker always have a long time STW while young gc, two or three times per day.


I am not sure this issue caused by Kafka or JVM itself.


Here is the GC log while the STW occured on the bottom of this mail， take a look at the row 344206 &nbsp;, it says &nbsp;the size of eden space changed from 30G to 3G


I want to figure out : Is this issue caused by my configuration of this broker? How can I fix it ?


or may be I just need send this log to Oracle


Thank you guys


Best Wishes





344181 2020-05-08T22:46:52.657+0800: 994003.630: [GC pause (G1 Evacuation Pause) (young), 5.6019987 secs]
344182 &nbsp; &nbsp;[Parallel Time: 5594.0 ms, GC Workers: 28]
344183 &nbsp; &nbsp; &nbsp; [GC Worker Start (ms): Min: 994003629.7, Avg: 994003629.8, Max: 994003630.0, Diff: 0.3]
344184 &nbsp; &nbsp; &nbsp; [Ext Root Scanning (ms): Min: 0.3, Avg: 0.4, Max: 0.6, Diff: 0.4, Sum: 11.5]
344185 &nbsp; &nbsp; &nbsp; [Update RS (ms): Min: 1.9, Avg: 2.2, Max: 2.4, Diff: 0.6, Sum: 60.2]
344186 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;[Processed Buffers: Min: 3, Avg: 4.0, Max: 9, Diff: 6, Sum: 111]
344187 &nbsp; &nbsp; &nbsp; [Scan RS (ms): Min: 0.0, Avg: 0.1, Max: 0.3, Diff: 0.3, Sum: 3.3]
344188 &nbsp; &nbsp; &nbsp; [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1]
344189 &nbsp; &nbsp; &nbsp; [Object Copy (ms): Min: 5463.1, Avg: 5484.2, Max: 5590.4, Diff: 127.3, Sum: 153558.5]
344190 &nbsp; &nbsp; &nbsp; [Termination (ms): Min: 0.0, Avg: 106.1, Max: 127.8, Diff: 127.8, Sum: 2971.9]
344191 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;[Termination Attempts: Min: 1, Avg: 1.4, Max: 2, Diff: 1, Sum: 39]
344192 &nbsp; &nbsp; &nbsp; [GC Worker Other (ms): Min: 0.0, Avg: 0.1, Max: 0.2, Diff: 0.2, Sum: 2.6]
344193 &nbsp; &nbsp; &nbsp; [GC Worker Total (ms): Min: 5592.9, Avg: 5593.1, Max: 5593.8, Diff: 0.9, Sum: 156608.2]
344194 &nbsp; &nbsp; &nbsp; [GC Worker End (ms): Min: 994009222.9, Avg: 994009223.0, Max: 994009223.6, Diff: 0.7]
344195 &nbsp; &nbsp;[Code Root Fixup: 0.1 ms]
344196 &nbsp; &nbsp;[Code Root Purge: 0.0 ms]
344197 &nbsp; &nbsp;[Clear CT: 3.0 ms]
344198 &nbsp; &nbsp;[Other: 4.9 ms]
344199 &nbsp; &nbsp; &nbsp; [Choose CSet: 0.0 ms]
344200 &nbsp; &nbsp; &nbsp; [Ref Proc: 2.4 ms]
344201 &nbsp; &nbsp; &nbsp; [Ref Enq: 0.1 ms]
344202 &nbsp; &nbsp; &nbsp; [Redirty Cards: 0.3 ms]
344203 &nbsp; &nbsp; &nbsp; [Humongous Register: 0.1 ms]
344204 &nbsp; &nbsp; &nbsp; [Humongous Reclaim: 0.0 ms]
344205 &nbsp; &nbsp; &nbsp; [Free CSet: 1.8 ms]
344206 &nbsp; &nbsp;[Eden: 30.4G(30.4G)-&gt;0.0B(3168.0M) Survivors: 128.0M-&gt;96.0M Heap: 50.6G(64.0G)-&gt;20.1G(64.0G)]
344207 &nbsp;[Times: user=0.00 sys=158.29, real=5.60 secs]