You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Prachi Garg <pg...@gridgain.com> on 2018/03/02 04:20:04 UTC
Enabling swap space and Ignite Persistence
Engineers,
How does persistence and swap work when both are enabled? I was under the
impression that for a data region you can either have swap or persistence
configured at a time, but not both. Please clarify.
Thanks,
-Prachi
Re: Enabling swap space and Ignite Persistence
Posted by Prachi Garg <pg...@gridgain.com>.
Created JIRA ticket for this issue -
https://issues.apache.org/jira/browse/IGNITE-7902
On Fri, Mar 2, 2018 at 10:34 AM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> I stumbled across couple of use cases where swap space was more suitable
> than persistence. However, enabling both for a same region definitely
> doesn't make sense to me, I would throw an exception in this case.
>
> -Val
>
> On Fri, Mar 2, 2018 at 9:46 AM, Denis Magda <dm...@apache.org> wrote:
>
> > Hi Ivan,
> >
> >
> > > Swap is legacy lightweight version of Ignite Native Persistence. In
> swap
> > > mode, we fully rely on OS in storing offheap memory into memory-mapped
> > > file. We don't provide durability guarantees in this mode. From my
> point
> > of
> > > view, after 2.1 release there's no reason to prefer swap mode over
> Ignite
> > > Native Persistence.
> > > Igniters, please correct me if there are still any actual cases.
> >
> >
> > There is a business use case for the swap space I've come across with
> > recently. Some applications want to store data entirely in RAM avoiding
> any
> > persistence in general (Ignite persistence or 3rd party DB). It's ok for
> > them to lose a data set in case of a cluster shutdown. However, they want
> > to avoid OOM exception that might happen if they don't scale out the
> > cluster in time. And here is the swap space comes to rescue. If a node is
> > running out of RAM, the OS begins the swap-out/in the process putting off
> > OOM and DevOps will have much more time to scale the cluster and
> rebalance
> > the data.
> >
> > Right now we indeed can configure both Ignite Native Persistence and
> > > swapping, but this makes even less sense. Node will just perform extra
> > job
> > > by persisting data twice.
> >
> >
> > Guess, that's the point Prachi tried to point out. Could we throw an
> > exception if a user tries to configure both? As we agreed, it's
> error-prone
> > and doesn't make sense in general.
> >
> > --
> > Denis
> >
> > On Fri, Mar 2, 2018 at 1:28 AM, Ivan Rakov <iv...@gmail.com>
> wrote:
> >
> > > Prachi,
> > >
> > > Swap is legacy lightweight version of Ignite Native Persistence. In
> swap
> > > mode, we fully rely on OS in storing offheap memory into memory-mapped
> > > file. We don't provide durability guarantees in this mode. From my
> point
> > of
> > > view, after 2.1 release there's no reason to prefer swap mode over
> Ignite
> > > Native Persistence.
> > > Igniters, please correct me if there are still any actual cases.
> > >
> > > Right now we indeed can configure both Ignite Native Persistence and
> > > swapping, but this makes even less sense. Node will just perform extra
> > job
> > > by persisting data twice.
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> > > On 02.03.2018 7:20, Prachi Garg wrote:
> > >
> > >> Engineers,
> > >>
> > >> How does persistence and swap work when both are enabled? I was under
> > the
> > >> impression that for a data region you can either have swap or
> > persistence
> > >> configured at a time, but not both. Please clarify.
> > >>
> > >> Thanks,
> > >> -Prachi
> > >>
> > >>
> > >
> >
>
Re: Enabling swap space and Ignite Persistence
Posted by Valentin Kulichenko <va...@gmail.com>.
I stumbled across couple of use cases where swap space was more suitable
than persistence. However, enabling both for a same region definitely
doesn't make sense to me, I would throw an exception in this case.
-Val
On Fri, Mar 2, 2018 at 9:46 AM, Denis Magda <dm...@apache.org> wrote:
> Hi Ivan,
>
>
> > Swap is legacy lightweight version of Ignite Native Persistence. In swap
> > mode, we fully rely on OS in storing offheap memory into memory-mapped
> > file. We don't provide durability guarantees in this mode. From my point
> of
> > view, after 2.1 release there's no reason to prefer swap mode over Ignite
> > Native Persistence.
> > Igniters, please correct me if there are still any actual cases.
>
>
> There is a business use case for the swap space I've come across with
> recently. Some applications want to store data entirely in RAM avoiding any
> persistence in general (Ignite persistence or 3rd party DB). It's ok for
> them to lose a data set in case of a cluster shutdown. However, they want
> to avoid OOM exception that might happen if they don't scale out the
> cluster in time. And here is the swap space comes to rescue. If a node is
> running out of RAM, the OS begins the swap-out/in the process putting off
> OOM and DevOps will have much more time to scale the cluster and rebalance
> the data.
>
> Right now we indeed can configure both Ignite Native Persistence and
> > swapping, but this makes even less sense. Node will just perform extra
> job
> > by persisting data twice.
>
>
> Guess, that's the point Prachi tried to point out. Could we throw an
> exception if a user tries to configure both? As we agreed, it's error-prone
> and doesn't make sense in general.
>
> --
> Denis
>
> On Fri, Mar 2, 2018 at 1:28 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> > Prachi,
> >
> > Swap is legacy lightweight version of Ignite Native Persistence. In swap
> > mode, we fully rely on OS in storing offheap memory into memory-mapped
> > file. We don't provide durability guarantees in this mode. From my point
> of
> > view, after 2.1 release there's no reason to prefer swap mode over Ignite
> > Native Persistence.
> > Igniters, please correct me if there are still any actual cases.
> >
> > Right now we indeed can configure both Ignite Native Persistence and
> > swapping, but this makes even less sense. Node will just perform extra
> job
> > by persisting data twice.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 02.03.2018 7:20, Prachi Garg wrote:
> >
> >> Engineers,
> >>
> >> How does persistence and swap work when both are enabled? I was under
> the
> >> impression that for a data region you can either have swap or
> persistence
> >> configured at a time, but not both. Please clarify.
> >>
> >> Thanks,
> >> -Prachi
> >>
> >>
> >
>
Re: Enabling swap space and Ignite Persistence
Posted by Denis Magda <dm...@apache.org>.
Hi Ivan,
> Swap is legacy lightweight version of Ignite Native Persistence. In swap
> mode, we fully rely on OS in storing offheap memory into memory-mapped
> file. We don't provide durability guarantees in this mode. From my point of
> view, after 2.1 release there's no reason to prefer swap mode over Ignite
> Native Persistence.
> Igniters, please correct me if there are still any actual cases.
There is a business use case for the swap space I've come across with
recently. Some applications want to store data entirely in RAM avoiding any
persistence in general (Ignite persistence or 3rd party DB). It's ok for
them to lose a data set in case of a cluster shutdown. However, they want
to avoid OOM exception that might happen if they don't scale out the
cluster in time. And here is the swap space comes to rescue. If a node is
running out of RAM, the OS begins the swap-out/in the process putting off
OOM and DevOps will have much more time to scale the cluster and rebalance
the data.
Right now we indeed can configure both Ignite Native Persistence and
> swapping, but this makes even less sense. Node will just perform extra job
> by persisting data twice.
Guess, that's the point Prachi tried to point out. Could we throw an
exception if a user tries to configure both? As we agreed, it's error-prone
and doesn't make sense in general.
--
Denis
On Fri, Mar 2, 2018 at 1:28 AM, Ivan Rakov <iv...@gmail.com> wrote:
> Prachi,
>
> Swap is legacy lightweight version of Ignite Native Persistence. In swap
> mode, we fully rely on OS in storing offheap memory into memory-mapped
> file. We don't provide durability guarantees in this mode. From my point of
> view, after 2.1 release there's no reason to prefer swap mode over Ignite
> Native Persistence.
> Igniters, please correct me if there are still any actual cases.
>
> Right now we indeed can configure both Ignite Native Persistence and
> swapping, but this makes even less sense. Node will just perform extra job
> by persisting data twice.
>
> Best Regards,
> Ivan Rakov
>
>
> On 02.03.2018 7:20, Prachi Garg wrote:
>
>> Engineers,
>>
>> How does persistence and swap work when both are enabled? I was under the
>> impression that for a data region you can either have swap or persistence
>> configured at a time, but not both. Please clarify.
>>
>> Thanks,
>> -Prachi
>>
>>
>
Re: Enabling swap space and Ignite Persistence
Posted by Ivan Rakov <iv...@gmail.com>.
Prachi,
Swap is legacy lightweight version of Ignite Native Persistence. In swap
mode, we fully rely on OS in storing offheap memory into memory-mapped
file. We don't provide durability guarantees in this mode. From my point
of view, after 2.1 release there's no reason to prefer swap mode over
Ignite Native Persistence.
Igniters, please correct me if there are still any actual cases.
Right now we indeed can configure both Ignite Native Persistence and
swapping, but this makes even less sense. Node will just perform extra
job by persisting data twice.
Best Regards,
Ivan Rakov
On 02.03.2018 7:20, Prachi Garg wrote:
> Engineers,
>
> How does persistence and swap work when both are enabled? I was under the
> impression that for a data region you can either have swap or persistence
> configured at a time, but not both. Please clarify.
>
> Thanks,
> -Prachi
>