How to set Expiry Policies when using Dataframe API to save data to Ignite?

[Ray <ra...@cisco.com>]

According to the document
https://apacheignite.readme.io/docs/expiry-policies, the expiry polices have
to be setup in the CacheConfiguration.
In the old RDD api, I can pass the CacheConfiguration to get a Ignite rdd
with expiry polices.
But in the Dataframe API, I don't see the expiry settings exists in the
IgniteDataFrameSettings.

So my question is how to set Expiry Policies when using Dataframe API to
save data to Ignite?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to set Expiry Policies when using Dataframe API to save data to Ignite?

[vkulichenko <va...@gmail.com>]

Ray,

You can use IgniteDataFrameSettings#OPTION_CREATE_TABLE_PARAMETERS parameter
to specify custom parameters that Spark should use when creating new table
in Ignite. Provided string will be attached to WITH clause of Ignite's
CREATE TABLE command [1].

For expiry policy in particular, you can have a predefined template with the
policy defined.

[1] https://apacheignite-sql.readme.io/docs/create-table

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to set Expiry Policies when using Dataframe API to save data to Ignite?

[dkarachentsev <dk...@gridgain.com>]

Hi Ray,

I think the only way to do it is to use
IgniteDataFrameSettings.OPTION_CONFIG_FILE and set path to xml configuration
with all settings you need. Here is a nice article about this [1]

[1]
https://medium.com/hashmapinc/apache-ignite-using-a-memory-grid-for-distributed-computation-frameworks-spark-and-flink-88de62417839

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/