Understanding Swap Space feature-- how it works?

[edwardk <ek...@gmail.com>]

Hi,

I am trying to understand the support in Ignite for spilling over to disk
when memory limits are reached.

I see there is a feature of swap space as in the link below,
https://apacheignite.readme.io/docs/off-heap-memory#swap-space

But, I am not clear on how it works.

When we configure swap-space it appears the cached data would be stored in
swap space.

But, what would the change be from the perspective of client. Is it
completely transparent to the client.

I do understand that there would be a performance impact if it is enabled,
as it would slow the cache operations as some data would be in memory and
some in disk.

Does this feature work and can we rely on it for critical scenarios where we
expect data size to be more which cannot be accommodated within the
configured memory limits.

Also, is there a way we can configure it per cache and not globally so that
for some items alone it can spill over to disk





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Understanding-Swap-Space-feature-how-it-works-tp10859.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Understanding Swap Space feature-- how it works?

[Ilya Kasnacheev <il...@gmail.com>]

Hello!

As far as my understanding goes, you do not need to do anything special to
get data off swap. You can just access any data in the cache, whether
swapped out or not.

I'm not sure how it would interact with eviction.

Regards,
-- 
Ilya Kasnacheev


ср, 24 мар. 2021 г. в 10:26, martinlundaskoe <ma...@gmail.com>:

> Hi Valentin
>
> TL/DR; How is data evicted from swap then?
>
> I know this is an old issue, but I am also trying to understand the swap
> feature.
>
> This may be outdated by now due to the addition of Native Persistence, but
> I
> have a use case where having swap instead of native persistence would be
> ideal.
>
> I first thought that the off-heap storage was defined per dataregion, and
> the data retained in RAM would be limited by the XX:MaxDirectMemorySize,
> the
> rest would be written to swap.
>
> Now I al really confused, so now my question is now, how to I get the data
> off swap?
> I don't see anything related to eviction of data from the cache that was
> written to swap... plus does it even make sense to have evicted data in
> swap?
>
>
> vkulichenko wrote
> > Basically, swap space is another storage layer. You have on-heap,
> off-heap
> > and then swap. So in case of on-heap mode only entry will be written in
> > swap when it's evicted from cache due to eviction policy. With off-heap
> > enabled it will be written to swap when it's evicted from there due to
> > off-heap memory limit.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Understanding Swap Space feature-- how it works?

[martinlundaskoe <ma...@gmail.com>]

Hi Valentin

TL/DR; How is data evicted from swap then?

I know this is an old issue, but I am also trying to understand the swap
feature.

This may be outdated by now due to the addition of Native Persistence, but I
have a use case where having swap instead of native persistence would be
ideal.

I first thought that the off-heap storage was defined per dataregion, and
the data retained in RAM would be limited by the XX:MaxDirectMemorySize, the
rest would be written to swap.

Now I al really confused, so now my question is now, how to I get the data
off swap?
I don't see anything related to eviction of data from the cache that was
written to swap... plus does it even make sense to have evicted data in
swap?


vkulichenko wrote
> Basically, swap space is another storage layer. You have on-heap, off-heap
> and then swap. So in case of on-heap mode only entry will be written in
> swap when it's evicted from cache due to eviction policy. With off-heap
> enabled it will be written to swap when it's evicted from there due to
> off-heap memory limit.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Understanding Swap Space feature-- how it works?

[vkulichenko <va...@gmail.com>]

Basically, swap space is another storage layer. You have on-heap, off-heap
and then swap. So in case of on-heap mode only entry will be written in swap
when it's evicted from cache due to eviction policy. With off-heap enabled
it will be written to swap when it's evicted from there due to off-heap
memory limit.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Understanding-Swap-Space-feature-how-it-works-tp10859p13033.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Understanding Swap Space feature-- how it works?

[rushi_rashi <ru...@gmail.com>]

Hi,
I have question related to swap memory.
If we have set setSwapEnabled to true, 
a. When will be the data is written to disc?
b. If by default data is written to disc only when free space is not
available in RAM, then can we limit the usage of RAM, lets say write to disc
when 80% of RAM is used. Although eviction policy writes data to off heap
memory but not to disc. 



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Understanding-Swap-Space-feature-how-it-works-tp10859p12968.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Understanding Swap Space feature-- how it works?

[vkulichenko <va...@gmail.com>]

Hi,

Yes, swap space is transparent. Only keep in mind that whenever you read an
entry from swap, it has to be unswapped first. So it's useful for the data
that is rarely used, but still can't be accessed sometimes. If all you data
is active, you will end up with a lot of swaps/unswaps which will seriously
drop performance.

Swap can be enabled or disabled on per cache level via
CacheConfiguration.setSwapEnabled property.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Understanding-Swap-Space-feature-how-it-works-tp10859p10885.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.