Cache Configuration for a data region

[the_palakkaran <ji...@suntecsbs.com>]

Hi,

I have multiple on heap caches attached to a data region of maximum 1GB size
durable memory. I have 512MB heap memory specified also.

Is it possible for me to set the size of this cache to take maximum of 100MB
and store the remaining in durable memory? Otherwise, won't it be using the
entire 2GB if new entries are put into it?

Or can this only be done using eviction policies?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

["slava.koptilin" <sl...@gmail.com>]

Hello,

> Do I need to configure something else for off heap eviction? 
If you are using Ignite Native Persistence, you don't need to configure
page-based eviction.
In that case, it will be automatically enabled. Please take a look at this
page for the details:
https://apacheignite.readme.io/docs/evictions#section-off-heap-memory

> As per the thread below, I need to set some data page eviction too, but 
> official documentation says it is deprecated
If I am not mistaken, it is not deprecated. At least I did not find any
mention of it.

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

[the_palakkaran <ji...@suntecsbs.com>]

Do I need to configure something else for off heap eviction?

I have on heap enabled and set eviction policy(LRU). Max size of 1MB also
provided. I have 1.5million data, so obviously it some entries loaded in the
start should be evicted. Still when I try to get those entries, I get
performance equivalent to the cache on heap reads. Is this normal or am I
missing something?

As per the thread below, I need to set some data page eviction too, but
official documentation says it is deprecated,

http://apache-ignite-users.70518.x6.nabble.com/Quick-questions-on-Evictions-td16632.html



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

["slava.koptilin" <sl...@gmail.com>]

> I guess durable memory is pure RAM based and native persistence is
combination of both RAM and disk.
Durable memory is a memory architecture that allows processing and storing
data both in memory and on disk. In other words, Ignite native persistence
is a feature(add-on) provided by Durable memory architecture.

> How to manipulate durable memory? Can I configure it?
Please take a look on this page:
https://apacheignite.readme.io/docs/memory-configuration

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

[the_palakkaran <ji...@suntecsbs.com>]

Thanks, every thing other than native vs durable is now clear for me.

I guess durable memory is pure RAM based and native persistence is
combination of both RAM and disk.

How to manipulate durable memory? Can I configure it?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

["slava.koptilin" <sl...@gmail.com>]

Hi,

> 1. I have set setOnheapCacheEnabled(true) for every cache.
> This means they will go into java heap right? 
As of AI 2.x Java heap is no longer treated as a data storage
and might be used as an extra caching layer for entries you have in the
off-heap.

> 2. I don't clearly understand the difference between durable memory and
> persistence.
Long story short, the 'Durable' memory is a page based memory architecture
that is split into pages of fixed size. Ignite 'Native persistence' is a
feature provided by Ignite and which allows storing your data on disk.
Please take a look at the following pages [1] & [2]

> 3. If I have enabled persistence and also set LRU eviction policy
> correctly,
> does this mean during loading also this eviction policy will work
> and I would never get an out of memory error? 
Yes, that is correct as long as you have free space on disk, of course.

> 4 How to check if the read happened from ignite persistence(disk) or
> memory?
There is no such capability if I am not mistaken. 

[1] https://apacheignite.readme.io/docs/durable-memory
[2] https://apacheignite.readme.io/docs/distributed-persistent-store

Thanks,
S.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

[the_palakkaran <ji...@suntecsbs.com>]

Hi,

1. I have set setOnheapCacheEnabled(true) for every cache. This means they
will go into java heap right?

2. Is it possible for me to take control of this durable memory? What should
essentially be kept there? How to do that? I don't clearly understand the
difference between durable memory and persistence.

3. If I have enabled persistence and also set LRU eviction policy correctly,
does this mean during loading also this eviction policy will work and I
would never get an out of memory error?

4. How to check if the read happened from ignite persistence(disk) or
memory?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Cache Configuration for a data region

["slava.koptilin" <sl...@gmail.com>]

Hello,

As of Apache Ignite 2.x, all data are stored into off-heap memory [1].
On-heap memory is used only for temporary operations, buffers etc.

> Also, I have my own cache store implementations. I hope this is only used
> for read through and
> write through from and to database and not for loading or writing to
> ignite durable memory.
> Can someone confirm this too?
Yes, that is correct.

> Again is it possible to restrict the number of entries kept in the memory
> while loading data into cache ?
> The rest can be kept in the durable (disk) memory.
I think that you need to configure eviction policy [2] and use Ignite Native
Persistence [3].
Please take into account that 'Durable' [4] memory is not the synonym of
'Native Persistence'.

[1]
https://apacheignite.readme.io/v2.5/docs/durable-memory#section-in-memory-features
[2] https://apacheignite.readme.io/docs/evictions
[3] https://apacheignite.readme.io/v2.5/docs/distributed-persistent-store
[4] https://apacheignite.readme.io/v2.5/docs/durable-memory

Thanks!




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/