You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Harshal Patil <ha...@mindtickle.com> on 2019/03/22 08:36:40 UTC

Spark Ignite SaveValues(rdd) not updating cache .

Hi ,
I am writing spark rdd to Ignie Cache .

If I do ,

*val *df = ic.fromCache("EntitySettingsCache").sql("select * from
entity_settings”).withColumn("partitionkey",*col*("partitionkey")/2)


ic.fromCache("EntitySettingsCache").saveValues(df.*rdd*)


*Not able to reflect changes in cache* ,


but i do ,


val rddPair: RDD[EntitySettingsKey , EntitySettings] = converRddToPair(rdd)


ic.fromCache("EntitySettingsCache").savePairs(df.*rdd , true*)


above code is relfecting changes in Cache .


If I see implementation of  *saveValues* and *savePairs* ,

there is one difference ,


*streamer.allowOverwrite(overwrite)*


above line is missing in saveValues , can anyone please help me , how do I
solve the problem , because I dont' want to convert *rdd* to *rddPair* manually
everytime .

Re: Spark Ignite SaveValues(rdd) not updating cache .

Posted by Harshal Patil <ha...@mindtickle.com>.

Can someone please help here ?

On Fri, Mar 22, 2019, 2:06 PM Harshal Patil <ha...@mindtickle.com>
wrote:

> Hi ,
> I am writing spark rdd to Ignie Cache .
>
> If I do ,
>
> *val *df = ic.fromCache("EntitySettingsCache").sql("select * from
> entity_settings”).withColumn("partitionkey",*col*("partitionkey")/2)
>
>
> ic.fromCache("EntitySettingsCache").saveValues(df.*rdd*)
>
>
> *Not able to reflect changes in cache* ,
>
>
> but i do ,
>
>
> val rddPair: RDD[EntitySettingsKey , EntitySettings] = converRddToPair(rdd)
>
>
> ic.fromCache("EntitySettingsCache").savePairs(df.*rdd , true*)
>
>
> above code is relfecting changes in Cache .
>
>
> If I see implementation of  *saveValues* and *savePairs* ,
>
> there is one difference ,
>
>
> *streamer.allowOverwrite(overwrite)*
>
>
> above line is missing in saveValues , can anyone please help me , how do I
> solve the problem , because I dont' want to convert *rdd* to *rddPair* manually
> everytime .
>
>
>
>
>
>
>
>
>
>
>