You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Adrian Mocanu <am...@verticalscope.com> on 2014/02/20 17:30:42 UTC

accumulator duration

How long are accumulators held in memory?

For example, if I run Spark non stop, and my data has its own timestamp on which I aggregate AND some of the data comes delayed 24hrs for a previous aggregation done 24hrs ago, can I still use accumulators? Or will they be garbage collected due to space... or are they saved somewhere when they are GC-ed.

-Adrian