You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jatinpreet <ja...@gmail.com> on 2015/03/17 16:27:49 UTC
High GC time
Hi,
I am getting very high GC time in my jobs. For smaller/real-time load, this
becomes a real problem.
Below are the details of a task I just ran. What could be the cause of such
skewed GC times?
36 2601 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:44 20 s 11 s
132.7 KB 135.8 KB
37 2602 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:44 15 s 11 s
79.4 KB 82.5 KB
38 2603 0 SUCCESS PROCESS_LOCAL 1 / Slave2 2015/03/17 11:18:44 2 s 0.7 s
0.0 B 37.8 KB
39 2604 0 SUCCESS PROCESS_LOCAL 0 / slave3 2015/03/17 11:18:45 21 s 18 s
77.9 KB 79.8 KB
40 2605 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:45 14 s 10 s
73.0 KB 74.9 KB
41 2606 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:45 14 s 10 s
74.4 KB 76.5 KB
42 2607 0 SUCCESS PROCESS_LOCAL 0 / Slave3 2015/03/17 11:18:45 12 s 12 s
10.9 KB 12.8 KB
Thanks
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/High-GC-time-tp22104.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: High GC time
Posted by Xiangrui Meng <me...@gmail.com>.
The official guide may help:
http://spark.apache.org/docs/latest/tuning.html#garbage-collection-tuning
-Xiangrui
On Tue, Mar 17, 2015 at 8:27 AM, jatinpreet <ja...@gmail.com> wrote:
> Hi,
>
> I am getting very high GC time in my jobs. For smaller/real-time load, this
> becomes a real problem.
>
> Below are the details of a task I just ran. What could be the cause of such
> skewed GC times?
>
> 36 2601 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:44 20 s 11 s
> 132.7 KB 135.8 KB
> 37 2602 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:44 15 s 11 s
> 79.4 KB 82.5 KB
> 38 2603 0 SUCCESS PROCESS_LOCAL 1 / Slave2 2015/03/17 11:18:44 2 s 0.7 s
> 0.0 B 37.8 KB
> 39 2604 0 SUCCESS PROCESS_LOCAL 0 / slave3 2015/03/17 11:18:45 21 s 18 s
> 77.9 KB 79.8 KB
> 40 2605 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:45 14 s 10 s
> 73.0 KB 74.9 KB
> 41 2606 0 SUCCESS PROCESS_LOCAL 2 / Slave1 2015/03/17 11:18:45 14 s 10 s
> 74.4 KB 76.5 KB
> 42 2607 0 SUCCESS PROCESS_LOCAL 0 / Slave3 2015/03/17 11:18:45 12 s 12 s
> 10.9 KB 12.8 KB
>
> Thanks
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/High-GC-time-tp22104.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org