You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Erik Selin <ty...@gmail.com> on 2014/01/03 05:19:07 UTC

cache eviction

Hey,

I was wondering how to evict a cached data set so to force it being rebuilt
from the underlying hdfs data. After reading the docs I'm still confused
how to ensure that intermediate cached result sets are evicted if the
underlying hdfs data changes. Are there any access points to force eviction
from the application layer?

Thanks!

Erik

Re: cache eviction

Posted by Sriram Ramachandrasekaran <sr...@gmail.com>.

RDD suports an unpersist(blocking: Boolean) method. It takes a boolean arg
which allows you to either remove the blocks in a blocking or async
fashion.

On Fri, Jan 3, 2014 at 9:49 AM, Erik Selin <ty...@gmail.com> wrote:

> Hey,
>
> I was wondering how to evict a cached data set so to force it being
> rebuilt from the underlying hdfs data. After reading the docs I'm still
> confused how to ensure that intermediate cached result sets are evicted if
> the underlying hdfs data changes. Are there any access points to force
> eviction from the application layer?
>
> Thanks!
>
> Erik
>

-- 
It's just about how deep your longing is!