Question About Persistence

[Alper Tekinalp <al...@evam.com>]

Hi everyone.

We have a cache that stores records with timestamp and regularly we run a
scan query like "timestamp <= now" and process returned records. As an
optimization we implemented such a logic that if timestamp of a record
greater than a threshold we don't put it to the cache but store directly to
the database. If it is within the threshold we put it to cache and with a
store adapter implementation we also put record to database. Because we
don't want to lose record through restarts.

In our current state, the store adapter is the bottleneck under load,
because of inserts and deletes. We insert new records and delete processed
ones.

So we thought persisting the data on file instead of database would perform
better. But at the same time we don't want to lose the threshold
optimization. So is it possible to implement such a logic with file
persistence that we only keep certain record on memory?

Do you have any suggestions that perform better in our use case?

Thanks a lot. Best regards.

-- 
Alper Tekinalp

Software Development Team Lead
Evam Streaming Analytics

www.evam.com <http://www.evam.com.tr>
<http://www.evam.com>

Re: Question About Persistence

[vkulichenko <va...@gmail.com>]

Hi Alper,

You can create two separate data regions [1], one for the data within the
threshold, another for outside. The latter can have very little memory
allocated and persistence enabled, which would mean that data in this region
is stored only on disc for the most part.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/