You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Mungeol Heo <mu...@gmail.com> on 2016/10/21 08:07:29 UTC

How to clean the accumulator and broadcast from the driver manually?

Hello,

As I mentioned at the title, I want to know is it possible to clean
the accumulator/broadcast from the driver manually since the driver's
memory keeps increasing.

Someone says that unpersist method removes them both from memory as
well as disk on each executor node. But it stays on the driver node,
so it can be re-broadcast

If it is true, how can I solve the "driver's memory keeps increasing" issue?

Any help will be GREAT!
Thank you.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org