You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Yair Ogen <ya...@gmail.com> on 2017/10/24 08:44:27 UTC
Ignite Spark - how to override and not append
I am using IgniteRdd to store and share RDD's between Spark jobs.
I notices that if I run the spark job over and over again it seems that
igniteContext.saveValues appends the values and doesn't behave like 'put'
in the direct cache API.
How can I work with IgniteRDD with override and not append?
Regards,
Yair
Re: Ignite Spark - how to override and not append
Posted by Николай Ижиков <ni...@gmail.com>.
Hello, Yair.
According to documentation:
`saveValues` works as follows: Saves values from given RDD into Ignite. *A
unique key will be generated for each value of the given RDD.*
I think, you can use `savePairs` with `overwrite` flag.
2017-10-24 11:44 GMT+03:00 Yair Ogen <ya...@gmail.com>:
> I am using IgniteRdd to store and share RDD's between Spark jobs.
>
> I notices that if I run the spark job over and over again it seems that
> igniteContext.saveValues appends the values and doesn't behave like 'put'
> in the direct cache API.
>
> How can I work with IgniteRDD with override and not append?
>
> Regards,
>
> Yair
>
--
Nikolay Izhikov
NIzhikov.dev@gmail.com