You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Harsha HN <99...@gmail.com> on 2015/08/18 09:15:56 UTC

Difference btw MEMORY_ONLY and MEMORY_AND_DISK

Hello Sparkers,

I would like to understand difference btw these Storage levels for a RDD
portion that doesn't fit in memory.
As it seems like in both storage levels, whatever portion doesnt fit in
memory will be spilled to disk. Any difference as such?

Thanks,
Harsha

Re: Difference btw MEMORY_ONLY and MEMORY_AND_DISK

Posted by Sabarish Sasidharan <sa...@manthan.com>.

MEMORY_ONLY will fail if there is not enough memory but MEMORY_AND_DISK
will spill to disk

Regards
Sab

On Tue, Aug 18, 2015 at 12:45 PM, Harsha HN <99...@gmail.com>
wrote:

> Hello Sparkers,
>
> I would like to understand difference btw these Storage levels for a RDD
> portion that doesn't fit in memory.
> As it seems like in both storage levels, whatever portion doesnt fit in
> memory will be spilled to disk. Any difference as such?
>
> Thanks,
> Harsha
>



-- 

Architect - Big Data
Ph: +91 99805 99458

Manthan Systems | *Company of the year - Analytics (2014 Frost and Sullivan
India ICT)*
+++