You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by vinod kumar <vi...@gmail.com> on 2015/10/09 10:47:43 UTC

Cache in Spark

Hi Guys,

May I know whether cache is enabled in spark by default?

Thanks,
Vinod

Re: Cache in Spark

Posted by Ted Yu <yu...@gmail.com>.

For RDD, I found this method:
  def getStorageLevel: StorageLevel = storageLevel

FYI

On Fri, Oct 9, 2015 at 2:46 AM, vinod kumar <vi...@gmail.com>
wrote:

> Thanks Natu,
>
> If so,Can you please share me the Spark SQL query to check whether the
> given table is cached or not? if you know
>
> Thanks,
> Vinod
>
> On Fri, Oct 9, 2015 at 2:26 PM, Natu Lauchande <nl...@gmail.com>
> wrote:
>
>>
>> I don't think so.
>>
>> Spark is not keeping the results in memory unless you tell it too.
>>
>> You have to explicitly call the cache method in your RDD:
>> linesWithSpark.cache()
>>
>> Thanks,
>> Natu
>>
>>
>>
>>
>> On Fri, Oct 9, 2015 at 10:47 AM, vinod kumar <vi...@gmail.com>
>> wrote:
>>
>>> Hi Guys,
>>>
>>> May I know whether cache is enabled in spark by default?
>>>
>>> Thanks,
>>> Vinod
>>>
>>
>>
>

Re: Cache in Spark

Posted by vinod kumar <vi...@gmail.com>.

Thanks Natu,

If so,Can you please share me the Spark SQL query to check whether the
given table is cached or not? if you know

Thanks,
Vinod

On Fri, Oct 9, 2015 at 2:26 PM, Natu Lauchande <nl...@gmail.com> wrote:

>
> I don't think so.
>
> Spark is not keeping the results in memory unless you tell it too.
>
> You have to explicitly call the cache method in your RDD:
> linesWithSpark.cache()
>
> Thanks,
> Natu
>
>
>
>
> On Fri, Oct 9, 2015 at 10:47 AM, vinod kumar <vi...@gmail.com>
> wrote:
>
>> Hi Guys,
>>
>> May I know whether cache is enabled in spark by default?
>>
>> Thanks,
>> Vinod
>>
>
>

Re: Cache in Spark

Posted by Natu Lauchande <nl...@gmail.com>.

I don't think so.

Spark is not keeping the results in memory unless you tell it too.

You have to explicitly call the cache method in your RDD:
linesWithSpark.cache()

Thanks,
Natu

On Fri, Oct 9, 2015 at 10:47 AM, vinod kumar <vi...@gmail.com>
wrote:

> Hi Guys,
>
> May I know whether cache is enabled in spark by default?
>
> Thanks,
> Vinod
>