You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Daniel Siegmann <da...@velos.io> on 2014/06/04 23:22:55 UTC
Re: How can I dispose an Accumulator?
Will the broadcast variables be disposed automatically if the context is
stopped, or do I still need to unpersist()?
On Sat, May 31, 2014 at 1:20 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey There,
>
> You can remove an accumulator by just letting it go out of scope and
> it will be garbage collected. For broadcast variables we actually
> store extra information for it, so we provide hooks for users to
> remove the associated state. There is no such need for accumulators,
> though.
>
> - Patrick
>
> On Thu, May 29, 2014 at 2:13 AM, innowireless TaeYun Kim
> <ta...@innowireless.co.kr> wrote:
> > Hi,
> >
> >
> >
> > How can I dispose an Accumulator?
> >
> > It has no method like 'unpersist()' which Broadcast provides.
> >
> >
> >
> > Thanks.
> >
> >
>
--
Daniel Siegmann, Software Developer
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
E: daniel.siegmann@velos.io W: www.velos.io
Re: How can I dispose an Accumulator?
Posted by Matei Zaharia <ma...@gmail.com>.
All of these are disposed of automatically if you stop the context or exit the program.
Matei
On Jun 4, 2014, at 2:22 PM, Daniel Siegmann <da...@velos.io> wrote:
> Will the broadcast variables be disposed automatically if the context is stopped, or do I still need to unpersist()?
>
>
> On Sat, May 31, 2014 at 1:20 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey There,
>
> You can remove an accumulator by just letting it go out of scope and
> it will be garbage collected. For broadcast variables we actually
> store extra information for it, so we provide hooks for users to
> remove the associated state. There is no such need for accumulators,
> though.
>
> - Patrick
>
> On Thu, May 29, 2014 at 2:13 AM, innowireless TaeYun Kim
> <ta...@innowireless.co.kr> wrote:
> > Hi,
> >
> >
> >
> > How can I dispose an Accumulator?
> >
> > It has no method like 'unpersist()' which Broadcast provides.
> >
> >
> >
> > Thanks.
> >
> >
>
>
>
> --
> Daniel Siegmann, Software Developer
> Velos
> Accelerating Machine Learning
>
> 440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
> E: daniel.siegmann@velos.io W: www.velos.io