You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by innowireless TaeYun Kim <ta...@innowireless.co.kr> on 2014/05/29 11:13:16 UTC
How can I dispose an Accumulator?
Hi,
How can I dispose an Accumulator?
It has no method like 'unpersist()' which Broadcast provides.
Thanks.
Re: How can I dispose an Accumulator?
Posted by Matei Zaharia <ma...@gmail.com>.
All of these are disposed of automatically if you stop the context or exit the program.
Matei
On Jun 4, 2014, at 2:22 PM, Daniel Siegmann <da...@velos.io> wrote:
> Will the broadcast variables be disposed automatically if the context is stopped, or do I still need to unpersist()?
>
>
> On Sat, May 31, 2014 at 1:20 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey There,
>
> You can remove an accumulator by just letting it go out of scope and
> it will be garbage collected. For broadcast variables we actually
> store extra information for it, so we provide hooks for users to
> remove the associated state. There is no such need for accumulators,
> though.
>
> - Patrick
>
> On Thu, May 29, 2014 at 2:13 AM, innowireless TaeYun Kim
> <ta...@innowireless.co.kr> wrote:
> > Hi,
> >
> >
> >
> > How can I dispose an Accumulator?
> >
> > It has no method like 'unpersist()' which Broadcast provides.
> >
> >
> >
> > Thanks.
> >
> >
>
>
>
> --
> Daniel Siegmann, Software Developer
> Velos
> Accelerating Machine Learning
>
> 440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
> E: daniel.siegmann@velos.io W: www.velos.io
Re: How can I dispose an Accumulator?
Posted by Daniel Siegmann <da...@velos.io>.
Will the broadcast variables be disposed automatically if the context is
stopped, or do I still need to unpersist()?
On Sat, May 31, 2014 at 1:20 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey There,
>
> You can remove an accumulator by just letting it go out of scope and
> it will be garbage collected. For broadcast variables we actually
> store extra information for it, so we provide hooks for users to
> remove the associated state. There is no such need for accumulators,
> though.
>
> - Patrick
>
> On Thu, May 29, 2014 at 2:13 AM, innowireless TaeYun Kim
> <ta...@innowireless.co.kr> wrote:
> > Hi,
> >
> >
> >
> > How can I dispose an Accumulator?
> >
> > It has no method like 'unpersist()' which Broadcast provides.
> >
> >
> >
> > Thanks.
> >
> >
>
--
Daniel Siegmann, Software Developer
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
E: daniel.siegmann@velos.io W: www.velos.io
Re: How can I dispose an Accumulator?
Posted by Patrick Wendell <pw...@gmail.com>.
Hey There,
You can remove an accumulator by just letting it go out of scope and
it will be garbage collected. For broadcast variables we actually
store extra information for it, so we provide hooks for users to
remove the associated state. There is no such need for accumulators,
though.
- Patrick
On Thu, May 29, 2014 at 2:13 AM, innowireless TaeYun Kim
<ta...@innowireless.co.kr> wrote:
> Hi,
>
>
>
> How can I dispose an Accumulator?
>
> It has no method like 'unpersist()' which Broadcast provides.
>
>
>
> Thanks.
>
>