why my YoungGen GC takes so long time?

[lisendong <li...@163.com>]

I found my task takes so long time for YoungGen GC, I set the young gen size
to about 1.5G, I wonder why it takes so long time?
not all the tasks take such long time, only about 1% tasks so long...


180.426: [GC [PSYoungGen: 9916105K->1676785K(14256640K)]
26201020K->18690057K(53403648K), 17.3581500 secs] [Times: user=104.48
sys=152.56, real=17.36 secs] 
198.986: [GC [PSYoungGen: 10724337K->1664837K(14881280K)]
27737609K->20205003K(54028288K), 55.0331460 secs] [Times: user=47.67
sys=776.67, real=55.02 secs] 
255.339: [GC [PSYoungGen: 11605317K->1550792K(14632960K)]
30145483K->21580959K(53779968K), 187.7893060 secs] [Times: user=110.10
sys=2704.33, real=187.76 secs] 
444.569: [GC [PSYoungGen: 11491272K->988109K(15197696K)]
31521439K->22370228K(54344704K), 366.2677820 secs] [Times: user=78.76
sys=5193.95, real=366.21 secs] 
812.450: [GC [PSYoungGen: 11704781K->1087351K(15092736K)]
33086900K->23295623K(54239744K), 163.0328770 secs] [Times: user=97.71
sys=2134.11, real=163.01 secs] 
977.207: [GC [PSYoungGen: 11804023K->1058228K(15470592K)]
34012295K->24294801K(54617600K), 176.5011980 secs] [Times: user=106.06
sys=1700.11, real=176.47 secs] 
1155.439: [GC [PSYoungGen: 12288948K->1248832K(15333888K)]
35525521K->25495802K(54480896K), 123.5496760 secs] [Times: user=59.09
sys=940.54, real=123.53 secs] 
1280.796: [GC [PSYoungGen: 12479552K->1097796K(15785472K)]
36726522K->26527988K(54932480K), 157.1727550 secs] [Times: user=11.14
sys=1376.93, real=157.15 secs] 
1439.923: [GC [PSYoungGen: 12953668K->1178863K(15644160K)]
38383860K->27637944K(54791168K), 119.3690200 secs] [Times: user=107.54
sys=937.40, real=119.35 secs] 
1561.236: [GC [PSYoungGen: 13034735K->1190849K(16093696K)]
39493816K->28787207K(55240704K), 159.1091770 secs] [Times: user=237.16
sys=1653.59, real=159.08 secs] 
1722.406: [GC [PSYoungGen: 13651905K->1245193K(15941120K)]
41248263K->29995801K(55088128K), 168.0231330 secs] [Times: user=81.20
sys=2203.72, real=168.00 secs] 
1892.731: [GC [PSYoungGen: 13706249K->1161132K(16381952K)]
42456857K->31117445K(55528960K), 162.0598100 secs] [Times: user=126.67
sys=1689.32, real=162.04 secs] 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/why-my-YoungGen-GC-takes-so-long-time-tp21941.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org