You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by 38797715 <38...@qq.com> on 2020/07/24 03:15:47 UTC
Enabling swapPath causes invoking shutdown hook
Hi community,
When swapPath is configured in DataRegionConfiguration and maxSize is
greater than the physical memory (that is, swap space is enabled), if
the amount of data exceeds the physical memory, a node failure will
occur. The log is as follows:
[08:29:14,212][INFO][Thread-24][G] Invoking shutdown hook...
I think the node process may be killed by the OS. What parameters can be
adjusted?
Re: Enabling swapPath causes invoking shutdown hook
Posted by Denis Magda <dm...@apache.org>.
If Ignite can hold all the records in memory it won't be reading pages from
disk (the page replacement). Thus, it's expected that the 12GB performs
better than the 5GB case.
Btw, are you planning to reload the whole data set into Ignite on potential
cluster restarts? Each node loses a subset of the data located in the swap
space on restarts.
-
Denis
On Thu, Aug 13, 2020 at 7:36 PM 38797715 <38...@qq.com> wrote:
> Hi Denis,
>
> We did a test, in the same environment (8G RAM, 10G swap partition) and
> the same configuration (2G Heap, enable persistence, data volume is about
> 6G), the only difference is that the maxSize size is different, which is
> configured as 5G and 12G respectively. We found that the performance of the
> scenario with maxSize = 12G is better than that of the scenario with
> maxSize = 5G, and the write performance is improved by more than 10%.
>
> I suspect that if the memory data region is large enough to hold all the
> data, ignite's page replacement might not be enabled.
>
> Our test scenarios are limited and may not be convincing. However, I think
> that the lack of memory may be the norm. At this time, it may be a good
> practice to make full use of the swap mechanism of the OS, which takes up
> more disk space but achieves better performance.
> 在 2020/8/14 上午8:22, Denis Magda 写道:
>
> Ignite swapping is based on the swapping mechanism of the OS. So, you
> shouldn’t see any difference if enable the OS one directly some way.
>
> Generally, you should not use swapping of any form as a permanent
> persistence layer due to the performance penalty. Once the swapping kicks
> in, you should scale out your cluster and wait while the cluster rebalances
> a part of the data to a new node. When the rebalancing completes, the
> performance will be recovered and swapping won’t longer be needed.
>
> Denis
>
> On Thursday, August 13, 2020, 38797715 <38...@qq.com> wrote:
>
>> Hi,
>>
>> We retested and found that if we configured swapPath, as the amount of
>> data increased, the write speed was actually slower and slower. If the
>> amount of data is large, on average, it is much slower than the scenario
>> where native persistence is enabled and wal is disabled.
>>
>> In this way, the use of the swapPath property has no productive value,
>> maybe it was an early development function, and now it is a bit out of date.
>>
>> What I want to ask is, in the case of small physical memory, turning on
>> persistence, and then configuring a larger maxSize (using the swap
>> mechanism of the OS), is this a solution? In other words, the swap
>> mechanism of the OS and the page replacement of Ignite, which is better?
>> 在 2020/8/6 下午9:23, Ilya Kasnacheev 写道:
>>
>> Hello!
>>
>> I think the performance of swap space should be on par with persistence
>> with disabled WAL.
>>
>> You can submit suggested updates to the documentation if you like.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 5 авг. 2020 г. в 06:00, 38797715 <38...@qq.com>:
>>
>>> Hi Ilya,
>>>
>>> If so, there are two ways to implement ignite's swap space:
>>> 1. maxSize > physical memory, which will use the swap mechanism of the
>>> OS, can be used *vm.swappiness* Adjust.
>>> 2. Configure the *swapPath* property, which is implemented by Ignite
>>> itself, is independent of the OS and has no optimization parameters.
>>> There's a choice between these two models, right? Then I think there may
>>> be many problems in the description of the document. I hope you can check
>>> it again:
>>> https://apacheignite.readme.io/docs/swap-space
>>>
>>> After our initial testing, the performance of swap space is much better
>>> than native persistence, so I think this pattern is valuable in some
>>> scenarios.
>>> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>>>
>>> Hello!
>>>
>>> From the docs:
>>>
>>> To avoid this situation with the swapping capabilities, you need to :
>>>
>>> - Set maxSize = bigger_ than_RAM_size, in which case, the OS will
>>> take care of the swapping.
>>> - Enable swapping by setting the DataRegionConfiguration.swapPath
>>> property.
>>>
>>>
>>> I actually think these are either-or. You should either do the first
>>> (and configure OS swapping) or the second part.
>>>
>>> Having said that, I recommend setting proper Native Persistence instead.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38...@qq.com>:
>>>
>>>> Hi,
>>>>
>>>> https://apacheignite.readme.io/docs/swap-space
>>>>
>>>> According to the above document, if the physical memory is small, you
>>>> can solve this problem by opening the swap space,The specific method is to
>>>> configure maxSize to a larger value (i.e. larger than the physical memory),
>>>> and the swapPath property needs to be configured.
>>>>
>>>> But from the test results, the node is terminated.
>>>>
>>>> I think the correct result should be that even if the amount of data
>>>> exceeds the physical memory, the node should still be able to run normally,
>>>> but the data is exchanged to the disk.
>>>>
>>>> I want to know what parameters affect the behavior of this
>>>> configuration? *vm.swappiness* or others?
>>>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>>>
>>>> Hi,
>>>>
>>>> Can you please clarify your expectations? You expected that JVM process will
>>>> be killed instead of gracefully stopping? What you are going to achieve?
>>>>
>>>> BR,
>>>> Andrei
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>>
>
> --
> -
> Denis
>
>
Re: Enabling swapPath causes invoking shutdown hook
Posted by 38797715 <38...@qq.com>.
Hi Denis,
We did a test, in the same environment (8G RAM, 10G swap partition) and
the same configuration (2G Heap, enable persistence, data volume is
about 6G), the only difference is that the maxSize size is different,
which is configured as 5G and 12G respectively. We found that the
performance of the scenario with maxSize = 12G is better than that of
the scenario with maxSize = 5G, and the write performance is improved by
more than 10%.
I suspect that if the memory data region is large enough to hold all the
data, ignite's page replacement might not be enabled.
Our test scenarios are limited and may not be convincing. However, I
think that the lack of memory may be the norm. At this time, it may be a
good practice to make full use of the swap mechanism of the OS, which
takes up more disk space but achieves better performance.
在 2020/8/14 上午8:22, Denis Magda 写道:
> Ignite swapping is based on the swapping mechanism of the OS. So, you
> shouldn’t see any difference if enable the OS one directly some way.
>
> Generally, you should not use swapping of any form as a permanent
> persistence layer due to the performance penalty. Once the swapping
> kicks in, you should scale out your cluster and wait while the cluster
> rebalances a part of the data to a new node. When the rebalancing
> completes, the performance will be recovered and swapping won’t longer
> be needed.
>
> Denis
>
> On Thursday, August 13, 2020, 38797715 <38797715@qq.com
> <ma...@qq.com>> wrote:
>
> Hi,
>
> We retested and found that if we configured swapPath, as the
> amount of data increased, the write speed was actually slower and
> slower. If the amount of data is large, on average, it is much
> slower than the scenario where native persistence is enabled and
> wal is disabled.
>
> In this way, the use of the swapPath property has no productive
> value, maybe it was an early development function, and now it is a
> bit out of date.
>
> What I want to ask is, in the case of small physical memory,
> turning on persistence, and then configuring a larger maxSize
> (using the swap mechanism of the OS), is this a solution? In other
> words, the swap mechanism of the OS and the page replacement of
> Ignite, which is better?
>
> 在 2020/8/6 下午9:23, Ilya Kasnacheev 写道:
>> Hello!
>>
>> I think the performance of swap space should be on par with
>> persistence with disabled WAL.
>>
>> You can submit suggested updates to the documentation if you like.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 5 авг. 2020 г. в 06:00, 38797715 <38797715@qq.com
>> <ma...@qq.com>>:
>>
>> Hi Ilya,
>>
>> If so, there are two ways to implement ignite's swap space:
>> 1. maxSize > physical memory, which will use the swap
>> mechanism of the OS, can be used *vm.swappiness* Adjust.
>> 2. Configure the *swapPath* property, which is implemented by
>> Ignite itself, is independent of the OS and has no
>> optimization parameters.
>>
>> There's a choice between these two models, right? Then I
>> think there may be many problems in the description of the
>> document. I hope you can check it again:
>> https://apacheignite.readme.io/docs/swap-space
>> <https://apacheignite.readme.io/docs/swap-space>
>>
>> After our initial testing, the performance of swap space is
>> much better than native persistence, so I think this pattern
>> is valuable in some scenarios.
>>
>> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>>> Hello!
>>>
>>> From the docs:
>>>
>>> To avoid this situation with the swapping capabilities, you
>>> need to :
>>>
>>> * Set |maxSize = bigger_ than_RAM_size|, in which case,
>>> the OS will take care of the swapping.
>>> * Enable swapping by setting the
>>> |DataRegionConfiguration.swapPath| property.
>>>
>>>
>>> I actually think these are either-or. You should either do
>>> the first (and configure OS swapping) or the second part.
>>>
>>> Having said that, I recommend setting proper Native
>>> Persistence instead.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38797715@qq.com
>>> <ma...@qq.com>>:
>>>
>>> Hi,
>>>
>>> https://apacheignite.readme.io/docs/swap-space
>>> <https://apacheignite.readme.io/docs/swap-space>
>>>
>>> According to the above document, if the physical memory
>>> is small, you can solve this problem by opening the swap
>>> space,The specific method is to configure maxSize to a
>>> larger value (i.e. larger than the physical memory), and
>>> the swapPath property needs to be configured.
>>>
>>> But from the test results, the node is terminated.
>>>
>>> I think the correct result should be that even if the
>>> amount of data exceeds the physical memory, the node
>>> should still be able to run normally, but the data is
>>> exchanged to the disk.
>>>
>>> I want to know what parameters affect the behavior of
>>> this configuration? *vm.swappiness* or others?
>>>
>>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>>> Hi,
>>>>
>>>> Can you please clarify your expectations? You expected that JVM process will
>>>> be killed instead of gracefully stopping? What you are going to achieve?
>>>>
>>>> BR,
>>>> Andrei
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from:http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
>>>
>
>
> --
> -
> Denis
>
Re: Enabling swapPath causes invoking shutdown hook
Posted by Denis Magda <dm...@apache.org>.
Ignite swapping is based on the swapping mechanism of the OS. So, you
shouldn’t see any difference if enable the OS one directly some way.
Generally, you should not use swapping of any form as a permanent
persistence layer due to the performance penalty. Once the swapping kicks
in, you should scale out your cluster and wait while the cluster rebalances
a part of the data to a new node. When the rebalancing completes, the
performance will be recovered and swapping won’t longer be needed.
Denis
On Thursday, August 13, 2020, 38797715 <38...@qq.com> wrote:
> Hi,
>
> We retested and found that if we configured swapPath, as the amount of
> data increased, the write speed was actually slower and slower. If the
> amount of data is large, on average, it is much slower than the scenario
> where native persistence is enabled and wal is disabled.
>
> In this way, the use of the swapPath property has no productive value,
> maybe it was an early development function, and now it is a bit out of date.
>
> What I want to ask is, in the case of small physical memory, turning on
> persistence, and then configuring a larger maxSize (using the swap
> mechanism of the OS), is this a solution? In other words, the swap
> mechanism of the OS and the page replacement of Ignite, which is better?
> 在 2020/8/6 下午9:23, Ilya Kasnacheev 写道:
>
> Hello!
>
> I think the performance of swap space should be on par with persistence
> with disabled WAL.
>
> You can submit suggested updates to the documentation if you like.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 5 авг. 2020 г. в 06:00, 38797715 <38...@qq.com>:
>
>> Hi Ilya,
>>
>> If so, there are two ways to implement ignite's swap space:
>> 1. maxSize > physical memory, which will use the swap mechanism of the
>> OS, can be used *vm.swappiness* Adjust.
>> 2. Configure the *swapPath* property, which is implemented by Ignite
>> itself, is independent of the OS and has no optimization parameters.
>> There's a choice between these two models, right? Then I think there may
>> be many problems in the description of the document. I hope you can check
>> it again:
>> https://apacheignite.readme.io/docs/swap-space
>>
>> After our initial testing, the performance of swap space is much better
>> than native persistence, so I think this pattern is valuable in some
>> scenarios.
>> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>>
>> Hello!
>>
>> From the docs:
>>
>> To avoid this situation with the swapping capabilities, you need to :
>>
>> - Set maxSize = bigger_ than_RAM_size, in which case, the OS will
>> take care of the swapping.
>> - Enable swapping by setting the DataRegionConfiguration.swapPath
>> property.
>>
>>
>> I actually think these are either-or. You should either do the first (and
>> configure OS swapping) or the second part.
>>
>> Having said that, I recommend setting proper Native Persistence instead.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38...@qq.com>:
>>
>>> Hi,
>>>
>>> https://apacheignite.readme.io/docs/swap-space
>>>
>>> According to the above document, if the physical memory is small, you
>>> can solve this problem by opening the swap space,The specific method is to
>>> configure maxSize to a larger value (i.e. larger than the physical memory),
>>> and the swapPath property needs to be configured.
>>>
>>> But from the test results, the node is terminated.
>>>
>>> I think the correct result should be that even if the amount of data
>>> exceeds the physical memory, the node should still be able to run normally,
>>> but the data is exchanged to the disk.
>>>
>>> I want to know what parameters affect the behavior of this
>>> configuration? *vm.swappiness* or others?
>>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>>
>>> Hi,
>>>
>>> Can you please clarify your expectations? You expected that JVM process will
>>> be killed instead of gracefully stopping? What you are going to achieve?
>>>
>>> BR,
>>> Andrei
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>>
--
-
Denis
Re: Enabling swapPath causes invoking shutdown hook
Posted by 38797715 <38...@qq.com>.
Hi,
We retested and found that if we configured swapPath, as the amount of
data increased, the write speed was actually slower and slower. If the
amount of data is large, on average, it is much slower than the scenario
where native persistence is enabled and wal is disabled.
In this way, the use of the swapPath property has no productive value,
maybe it was an early development function, and now it is a bit out of date.
What I want to ask is, in the case of small physical memory, turning on
persistence, and then configuring a larger maxSize (using the swap
mechanism of the OS), is this a solution? In other words, the swap
mechanism of the OS and the page replacement of Ignite, which is better?
在 2020/8/6 下午9:23, Ilya Kasnacheev 写道:
> Hello!
>
> I think the performance of swap space should be on par with
> persistence with disabled WAL.
>
> You can submit suggested updates to the documentation if you like.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 5 авг. 2020 г. в 06:00, 38797715 <38797715@qq.com
> <ma...@qq.com>>:
>
> Hi Ilya,
>
> If so, there are two ways to implement ignite's swap space:
> 1. maxSize > physical memory, which will use the swap mechanism of
> the OS, can be used *vm.swappiness* Adjust.
> 2. Configure the *swapPath* property, which is implemented by
> Ignite itself, is independent of the OS and has no optimization
> parameters.
>
> There's a choice between these two models, right? Then I think
> there may be many problems in the description of the document. I
> hope you can check it again:
> https://apacheignite.readme.io/docs/swap-space
> <https://apacheignite.readme.io/docs/swap-space>
>
> After our initial testing, the performance of swap space is much
> better than native persistence, so I think this pattern is
> valuable in some scenarios.
>
> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>> Hello!
>>
>> From the docs:
>>
>> To avoid this situation with the swapping capabilities, you need to :
>>
>> * Set |maxSize = bigger_ than_RAM_size|, in which case, the OS
>> will take care of the swapping.
>> * Enable swapping by setting the
>> |DataRegionConfiguration.swapPath| property.
>>
>>
>> I actually think these are either-or. You should either do the
>> first (and configure OS swapping) or the second part.
>>
>> Having said that, I recommend setting proper Native Persistence
>> instead.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38797715@qq.com
>> <ma...@qq.com>>:
>>
>> Hi,
>>
>> https://apacheignite.readme.io/docs/swap-space
>> <https://apacheignite.readme.io/docs/swap-space>
>>
>> According to the above document, if the physical memory is
>> small, you can solve this problem by opening the swap
>> space,The specific method is to configure maxSize to a larger
>> value (i.e. larger than the physical memory), and the
>> swapPath property needs to be configured.
>>
>> But from the test results, the node is terminated.
>>
>> I think the correct result should be that even if the amount
>> of data exceeds the physical memory, the node should still be
>> able to run normally, but the data is exchanged to the disk.
>>
>> I want to know what parameters affect the behavior of this
>> configuration? *vm.swappiness* or others?
>>
>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>> Hi,
>>>
>>> Can you please clarify your expectations? You expected that JVM process will
>>> be killed instead of gracefully stopping? What you are going to achieve?
>>>
>>> BR,
>>> Andrei
>>>
>>>
>>>
>>> --
>>> Sent from:http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
>>
Re: Enabling swapPath causes invoking shutdown hook
Posted by Denis Magda <dm...@apache.org>.
Agree with Ilya that the performance should be comparable if you disable
the WAL of the Ignite persistence.
Anyway, the swapping and Ignite persistence pursue different goals. The
swapping is one of the out-of-memory protection techniques - if you run out
of DRAM, then the OS will start swapping in/out Ignite memory pages to
avoid a node outage. But, the swap space is not a durable storage layer. If
you restart the cluster, then all the swapped pages will evaporate. While,
Ignite persistence is your durable disk tier that survives cluster restarts
and, thus, uses more sophisticated algorithms to ensure data consistency
and durability. Just select what suits you best. I put more thoughts on
this in this article:
https://www.gridgain.com/resources/blog/out-of-memory-apache-ignite-cluster-handling-techniques
-
Denis
On Thu, Aug 6, 2020 at 6:23 AM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I think the performance of swap space should be on par with persistence
> with disabled WAL.
>
> You can submit suggested updates to the documentation if you like.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 5 авг. 2020 г. в 06:00, 38797715 <38...@qq.com>:
>
>> Hi Ilya,
>>
>> If so, there are two ways to implement ignite's swap space:
>> 1. maxSize > physical memory, which will use the swap mechanism of the
>> OS, can be used *vm.swappiness* Adjust.
>> 2. Configure the *swapPath* property, which is implemented by Ignite
>> itself, is independent of the OS and has no optimization parameters.
>> There's a choice between these two models, right? Then I think there may
>> be many problems in the description of the document. I hope you can check
>> it again:
>> https://apacheignite.readme.io/docs/swap-space
>>
>> After our initial testing, the performance of swap space is much better
>> than native persistence, so I think this pattern is valuable in some
>> scenarios.
>> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>>
>> Hello!
>>
>> From the docs:
>>
>> To avoid this situation with the swapping capabilities, you need to :
>>
>> - Set maxSize = bigger_ than_RAM_size, in which case, the OS will
>> take care of the swapping.
>> - Enable swapping by setting the DataRegionConfiguration.swapPath
>> property.
>>
>>
>> I actually think these are either-or. You should either do the first (and
>> configure OS swapping) or the second part.
>>
>> Having said that, I recommend setting proper Native Persistence instead.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> сб, 25 июл. 2020 г. в 04:49, 38797715 <38...@qq.com>:
>>
>>> Hi,
>>>
>>> https://apacheignite.readme.io/docs/swap-space
>>>
>>> According to the above document, if the physical memory is small, you
>>> can solve this problem by opening the swap space,The specific method is to
>>> configure maxSize to a larger value (i.e. larger than the physical memory),
>>> and the swapPath property needs to be configured.
>>>
>>> But from the test results, the node is terminated.
>>>
>>> I think the correct result should be that even if the amount of data
>>> exceeds the physical memory, the node should still be able to run normally,
>>> but the data is exchanged to the disk.
>>>
>>> I want to know what parameters affect the behavior of this
>>> configuration? *vm.swappiness* or others?
>>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>>
>>> Hi,
>>>
>>> Can you please clarify your expectations? You expected that JVM process will
>>> be killed instead of gracefully stopping? What you are going to achieve?
>>>
>>> BR,
>>> Andrei
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>>
Re: Enabling swapPath causes invoking shutdown hook
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think the performance of swap space should be on par with persistence
with disabled WAL.
You can submit suggested updates to the documentation if you like.
Regards,
--
Ilya Kasnacheev
ср, 5 авг. 2020 г. в 06:00, 38797715 <38...@qq.com>:
> Hi Ilya,
>
> If so, there are two ways to implement ignite's swap space:
> 1. maxSize > physical memory, which will use the swap mechanism of the OS,
> can be used *vm.swappiness* Adjust.
> 2. Configure the *swapPath* property, which is implemented by Ignite
> itself, is independent of the OS and has no optimization parameters.
> There's a choice between these two models, right? Then I think there may
> be many problems in the description of the document. I hope you can check
> it again:
> https://apacheignite.readme.io/docs/swap-space
>
> After our initial testing, the performance of swap space is much better
> than native persistence, so I think this pattern is valuable in some
> scenarios.
> 在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
>
> Hello!
>
> From the docs:
>
> To avoid this situation with the swapping capabilities, you need to :
>
> - Set maxSize = bigger_ than_RAM_size, in which case, the OS will take
> care of the swapping.
> - Enable swapping by setting the DataRegionConfiguration.swapPath
> property.
>
>
> I actually think these are either-or. You should either do the first (and
> configure OS swapping) or the second part.
>
> Having said that, I recommend setting proper Native Persistence instead.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> сб, 25 июл. 2020 г. в 04:49, 38797715 <38...@qq.com>:
>
>> Hi,
>>
>> https://apacheignite.readme.io/docs/swap-space
>>
>> According to the above document, if the physical memory is small, you can
>> solve this problem by opening the swap space,The specific method is to
>> configure maxSize to a larger value (i.e. larger than the physical memory),
>> and the swapPath property needs to be configured.
>>
>> But from the test results, the node is terminated.
>>
>> I think the correct result should be that even if the amount of data
>> exceeds the physical memory, the node should still be able to run normally,
>> but the data is exchanged to the disk.
>>
>> I want to know what parameters affect the behavior of this configuration?
>> *vm.swappiness* or others?
>> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>>
>> Hi,
>>
>> Can you please clarify your expectations? You expected that JVM process will
>> be killed instead of gracefully stopping? What you are going to achieve?
>>
>> BR,
>> Andrei
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
Re: Enabling swapPath causes invoking shutdown hook
Posted by 38797715 <38...@qq.com>.
Hi Ilya,
If so, there are two ways to implement ignite's swap space:
1. maxSize > physical memory, which will use the swap mechanism of the
OS, can be used *vm.swappiness* Adjust.
2. Configure the *swapPath* property, which is implemented by Ignite
itself, is independent of the OS and has no optimization parameters.
There's a choice between these two models, right? Then I think there may
be many problems in the description of the document. I hope you can
check it again:
https://apacheignite.readme.io/docs/swap-space
After our initial testing, the performance of swap space is much better
than native persistence, so I think this pattern is valuable in some
scenarios.
在 2020/8/4 下午10:16, Ilya Kasnacheev 写道:
> Hello!
>
> From the docs:
>
> To avoid this situation with the swapping capabilities, you need to :
>
> * Set |maxSize = bigger_ than_RAM_size|, in which case, the OS will
> take care of the swapping.
> * Enable swapping by setting the
> |DataRegionConfiguration.swapPath| property.
>
>
> I actually think these are either-or. You should either do the first
> (and configure OS swapping) or the second part.
>
> Having said that, I recommend setting proper Native Persistence instead.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> сб, 25 июл. 2020 г. в 04:49, 38797715 <38797715@qq.com
> <ma...@qq.com>>:
>
> Hi,
>
> https://apacheignite.readme.io/docs/swap-space
> <https://apacheignite.readme.io/docs/swap-space>
>
> According to the above document, if the physical memory is small,
> you can solve this problem by opening the swap space,The specific
> method is to configure maxSize to a larger value (i.e. larger than
> the physical memory), and the swapPath property needs to be
> configured.
>
> But from the test results, the node is terminated.
>
> I think the correct result should be that even if the amount of
> data exceeds the physical memory, the node should still be able to
> run normally, but the data is exchanged to the disk.
>
> I want to know what parameters affect the behavior of this
> configuration? *vm.swappiness* or others?
>
> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>> Hi,
>>
>> Can you please clarify your expectations? You expected that JVM process will
>> be killed instead of gracefully stopping? What you are going to achieve?
>>
>> BR,
>> Andrei
>>
>>
>>
>> --
>> Sent from:http://apache-ignite-users.70518.x6.nabble.com/ <http://apache-ignite-users.70518.x6.nabble.com/>
>
Re: Enabling swapPath causes invoking shutdown hook
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
From the docs:
To avoid this situation with the swapping capabilities, you need to :
- Set maxSize = bigger_ than_RAM_size, in which case, the OS will take
care of the swapping.
- Enable swapping by setting the DataRegionConfiguration.swapPath
property.
I actually think these are either-or. You should either do the first (and
configure OS swapping) or the second part.
Having said that, I recommend setting proper Native Persistence instead.
Regards,
--
Ilya Kasnacheev
сб, 25 июл. 2020 г. в 04:49, 38797715 <38...@qq.com>:
> Hi,
>
> https://apacheignite.readme.io/docs/swap-space
>
> According to the above document, if the physical memory is small, you can
> solve this problem by opening the swap space,The specific method is to
> configure maxSize to a larger value (i.e. larger than the physical memory),
> and the swapPath property needs to be configured.
>
> But from the test results, the node is terminated.
>
> I think the correct result should be that even if the amount of data
> exceeds the physical memory, the node should still be able to run normally,
> but the data is exchanged to the disk.
>
> I want to know what parameters affect the behavior of this configuration?
> *vm.swappiness* or others?
> 在 2020/7/24 下午9:55, aealexsandrov 写道:
>
> Hi,
>
> Can you please clarify your expectations? You expected that JVM process will
> be killed instead of gracefully stopping? What you are going to achieve?
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
Re: Enabling swapPath causes invoking shutdown hook
Posted by aealexsandrov <ae...@gmail.com>.
Hi,
Documentation looks not very clean for me. Probably you should prepare your
operating system to use swap space. Can you try to prepare the swap file
first as following:
https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-18-04/
And then choose this file.
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Enabling swapPath causes invoking shutdown hook
Posted by 38797715 <38...@qq.com>.
Hi,
https://apacheignite.readme.io/docs/swap-space
According to the above document, if the physical memory is small, you
can solve this problem by opening the swap space,The specific method is
to configure maxSize to a larger value (i.e. larger than the physical
memory), and the swapPath property needs to be configured.
But from the test results, the node is terminated.
I think the correct result should be that even if the amount of data
exceeds the physical memory, the node should still be able to run
normally, but the data is exchanged to the disk.
I want to know what parameters affect the behavior of this
configuration? *vm.swappiness* or others?
在 2020/7/24 下午9:55, aealexsandrov 写道:
> Hi,
>
> Can you please clarify your expectations? You expected that JVM process will
> be killed instead of gracefully stopping? What you are going to achieve?
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Enabling swapPath causes invoking shutdown hook
Posted by aealexsandrov <ae...@gmail.com>.
Hi,
Can you please clarify your expectations? You expected that JVM process will
be killed instead of gracefully stopping? What you are going to achieve?
BR,
Andrei
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/