You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Alexey Goncharuk <al...@gmail.com> on 2020/02/03 15:47:20 UTC
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
I do not mind making this change if we explicitly and clearly define what
'new inactive state' means. What should happen if a partition is lost in
inactive state? What if such node joins back the cluster after? Etc.
пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> Back up Ivan's opinion here. Initially, the activation/deactivation was
> created for the baseline topology designed for cases with native
> persistence. My thinking was that the mechanism itended to prevent data
> inconsistencies while nodes with data on the disk will be in the process of
> joining the cluster.
>
> Artem, could you please update the docs bringing this to the attention of
> the user community?
> https://issues.apache.org/jira/browse/IGNITE-12615
>
> AG, what if we don't purge data from memory at least for the caches not
> backed by the native persistence? Is this a big deal? We can certainly put
> this off by my guts feel we'll return to this question sooner or later.
>
> -
> Denis
>
>
> On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
> > For me it looks like some coincidence effect. I understand that we get
> > such behavior because deactivation works the same way as for
> > persistent caches. Was cluster activation/deactivation designed and
> > described for in-memory caches? Current behavior sounds for me a as
> > big risk. I expect a lot of upset users unexpectedly purged all their
> > data.
> >
> > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> > >
> > > Because originally the sole purpose of deactivation was resource
> > > deallocation.
> > >
> > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > >
> > > > Such a revelation for me that data is purged from RAM if someone
> > > > deactivates the cluster. Alex, do you remember why we decided to
> > implement
> > > > it this way initially?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com>
> > > > wrote:
> > > >
> > > > > I agree on CLI and JMX because those interfaces can be used by a
> > system
> > > > > administrator and can be invoked by mistake.
> > > > >
> > > > > As for the Java API, personally, I find it strange to add 'force'
> or
> > > > > 'confirm' flags to it because it is very unlikely that such an
> > invocation
> > > > > is done by mistake. Such mistakes are caught during the testing
> > phase and
> > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Hi, folks.
Another question has raised. Can I remove / rename / bring substitution for
IgniteMXBean#active(boolean active) ? Or at least to deprecate?
The problem is that it interferes with Ignite#active(boolean active). Same
signature for different interfaces. And only one implementation:
IgniteKernal#active(boolean). We decided not to touch internal calls to
cluster deactivation. We warn user only at manual deactivation via CLI and
JMX. In the code and tests one can find many usages of
Ignite#active(boolean active). I can’t, for example, throw an exception
from it. Several tests will fail. Including C++ and C# ones. But this also
means IgniteMXBean#active(boolean active) won’t warn user of the data loss
too. Adding new method to the JMX bean won’t solve the problem. Old
active(boolean) comes first, clearly seen and is obvious to use.
What is the best practice for cases like this?
пт, 7 февр. 2020 г. в 11:23, Nikolay Izhikov <ni...@apache.org>:
> > To prevent unexpected data loss on
> deactivation CLI and JMX should require 'force' flag
>
> Vladimir, please, go for it.
>
>
> пт, 7 февр. 2020 г., 11:19 Vladimir Steshin <vl...@gmail.com>:
>
> > Ok. Then we are at the beginning. To prevent unexpected data loss on
> > deactivation CLI and JMX should require 'force' flag. If there are no
> extra
> > proposals I'm going to finish the pull request.
> >
> > чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com
> > >:
> >
> > > Vladimir,
> > >
> > > IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> > > clearing.
> > > It allows allocated memory reuse on re-activation to avoid OS specific
> > > issues if allocated heap is rather large.
> > >
> > > You are right to create separate ticket for implementing required
> > behavior.
> > >
> > > чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
> > >
> > > > Or, is flag [1] is actual only for persistence mode? Related ticket
> [2]
> > > is
> > > > completely about disabled persistence.
> > > > If previous assumption is right, I think, we can involve flag [1] in
> > > ticket
> > > > [2].
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> > > >
> > > > > Denis, Alexei,
> > > > >
> > > > > Regarding usage of flag
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > > [1]
> > > > >
> > > > > When enabled, I think the following test should work. But it fails.
> > > > >
> > > > > //----------------------------------------------------------------
> > > > > @Test
> > > > > public void testDataPresent() throws Exception {
> > > > > System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE,
> > "true");
> > > > >
> > > > > IgniteEx i = startGrid(0);
> > > > >
> > > > > assertFalse(
> > > > >
> > > >
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > > > > .isPersistenceEnabled() );
> > > > >
> > > > > String name = "non-persistent-cache";
> > > > >
> > > > > i.createCache(name).put(1L, 1L);
> > > > >
> > > > > assertEquals(1L, i.cache(name).get(1L));
> > > > >
> > > > > i.cluster().state(ClusterState.INACTIVE);
> > > > > i.cluster().state(ClusterState.ACTIVE);
> > > > >
> > > > > assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> > > here!
> > > > > }
> > > > > //----------------------------------------------------------------
> > > > >
> > > > >
> > > > > Several notes:
> > > > >
> > > > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > > > > is true
> > > > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > > > > is called with shutdown == false
> > > > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > > > > is called with deallocate == false
> > > > >
> > > > > But the cache from the test still has zero size after reactivation.
> > > > >
> > > > > Is flag [1] disabled by default because it is not implemented /
> > doesn't
> > > > > work? Do we need to skip it in current ticket and rise new one?
> > > > >
> > > > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > > > >
> > > > >> I believe there might be a consistency-related reason why this
> flag
> > > was
> > > > >> disabled by default for caches that store data in Ignite native
> > > > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some
> > light
> > > on
> > > > >> this.
> > > > >>
> > > > >> As for the memory-only caches or caches backed up by a CacheStore
> > such
> > > > as
> > > > >> an RDBMS, enabling of the flag should be harmless. Once we do
> this,
> > > > we'll
> > > > >> eliminate the need to load the data back into the cluster which
> can
> > > be a
> > > > >> time-consuming operation depending on the data volume.
> > > > >>
> > > > >>
> > > > >> -
> > > > >> Denis
> > > > >>
> > > > >>
> > > > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <
> > vladsz83@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> > > There
> > > > >> > should be a reason for that. Any data consistency issues when
> node
> > > > gets
> > > > >> > activated anew? We may use both solutions because the flag can
> be
> > > > >> switched
> > > > >> > off again.
> > > > >> >
> > > > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > > > >> >
> > > > >> > > Hi Vladimir,
> > > > >> > >
> > > > >> > > Yes, I'm suggesting us to enable this flag by
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > default instead of introducing --force flag and showing any
> > > > warnings.
> > > > >> > >
> > > > >> > > -
> > > > >> > > Denis
> > > > >> > >
> > > > >> > >
> > > > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> > > vladsz83@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hello all.
> > > > >> > > >
> > > > >> > > > Denis, which changes exactly? In current implementation of
> > > ticket
> > > > >> [2]
> > > > >> > > flag
> > > > >> > > > [1] is checked before requiring --force flag and showing any
> > > > >> warnings.
> > > > >> > Do
> > > > >> > > > we need to set reuse-memory-on-deactivate to true by
> default?
> > > > >> > > >
> > > > >> > > > [1]
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > >
> > > > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dmagda@apache.org
> >:
> > > > >> > > >
> > > > >> > > > > That's the best solution for this scenario. Should we
> > readjust
> > > > the
> > > > >> > > > already
> > > > >> > > > > created ticket [1] suggesting to implement the changes of
> > Alex
> > > > >> > > Scherbakov
> > > > >> > > > > instead?
> > > > >> > > > >
> > > > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >> > > > >
> > > > >> > > > > -
> > > > >> > > > > Denis
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Folks,
> > > > >> > > > > >
> > > > >> > > > > > For a long time we have a flag [1]
> > > > >> > > > > >
> > > > >> > > > > > It does almost what we want here.
> > > > >> > > > > >
> > > > >> > > > > > I suggest to make this behavior default and rework it to
> > > keep
> > > > >> data
> > > > >> > in
> > > > >> > > > > > memory as well (we already have special "recovery" mode
> > for
> > > > >> > caches).
> > > > >> > > > > >
> > > > >> > > > > > [1]
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > >> > > > > alexey.goncharuk@gmail.com
> > > > >> > > > > > >:
> > > > >> > > > > >
> > > > >> > > > > > > I do not mind making this change if we explicitly and
> > > > clearly
> > > > >> > > define
> > > > >> > > > > what
> > > > >> > > > > > > 'new inactive state' means. What should happen if a
> > > > partition
> > > > >> is
> > > > >> > > lost
> > > > >> > > > > in
> > > > >> > > > > > > inactive state? What if such node joins back the
> cluster
> > > > >> after?
> > > > >> > > Etc.
> > > > >> > > > > > >
> > > > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> > > dmagda@apache.org
> > > > >:
> > > > >> > > > > > >
> > > > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > > >> > > activation/deactivation
> > > > >> > > > > was
> > > > >> > > > > > > > created for the baseline topology designed for cases
> > > with
> > > > >> > native
> > > > >> > > > > > > > persistence. My thinking was that the mechanism
> > itended
> > > to
> > > > >> > > prevent
> > > > >> > > > > data
> > > > >> > > > > > > > inconsistencies while nodes with data on the disk
> will
> > > be
> > > > in
> > > > >> > the
> > > > >> > > > > > process
> > > > >> > > > > > > of
> > > > >> > > > > > > > joining the cluster.
> > > > >> > > > > > > >
> > > > >> > > > > > > > Artem, could you please update the docs bringing
> this
> > to
> > > > the
> > > > >> > > > > attention
> > > > >> > > > > > of
> > > > >> > > > > > > > the user community?
> > > > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > >> > > > > > > >
> > > > >> > > > > > > > AG, what if we don't purge data from memory at least
> > for
> > > > the
> > > > >> > > caches
> > > > >> > > > > not
> > > > >> > > > > > > > backed by the native persistence? Is this a big
> deal?
> > We
> > > > can
> > > > >> > > > > certainly
> > > > >> > > > > > > put
> > > > >> > > > > > > > this off by my guts feel we'll return to this
> question
> > > > >> sooner
> > > > >> > or
> > > > >> > > > > later.
> > > > >> > > > > > > >
> > > > >> > > > > > > > -
> > > > >> > > > > > > > Denis
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > > >> > > > vololo100@gmail.com>
> > > > >> > > > > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > For me it looks like some coincidence effect. I
> > > > understand
> > > > >> > that
> > > > >> > > > we
> > > > >> > > > > > get
> > > > >> > > > > > > > > such behavior because deactivation works the same
> > way
> > > as
> > > > >> for
> > > > >> > > > > > > > > persistent caches. Was cluster
> > activation/deactivation
> > > > >> > designed
> > > > >> > > > and
> > > > >> > > > > > > > > described for in-memory caches? Current behavior
> > > sounds
> > > > >> for
> > > > >> > me
> > > > >> > > a
> > > > >> > > > as
> > > > >> > > > > > > > > big risk. I expect a lot of upset users
> unexpectedly
> > > > >> purged
> > > > >> > all
> > > > >> > > > > their
> > > > >> > > > > > > > > data.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > >> > > > > > > > alexey.goncharuk@gmail.com
> > > > >> > > > > > > > > >:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Because originally the sole purpose of
> > deactivation
> > > > was
> > > > >> > > > resource
> > > > >> > > > > > > > > > deallocation.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > > > >> > dmagda@apache.org
> > > > >> > > >:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > > Such a revelation for me that data is purged
> > from
> > > > RAM
> > > > >> if
> > > > >> > > > > someone
> > > > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember
> > why
> > > > we
> > > > >> > > decided
> > > > >> > > > > to
> > > > >> > > > > > > > > implement
> > > > >> > > > > > > > > > > it this way initially?
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > -
> > > > >> > > > > > > > > > > Denis
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey
> > Goncharuk <
> > > > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > >> > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > > I agree on CLI and JMX because those
> > interfaces
> > > > can
> > > > >> be
> > > > >> > > used
> > > > >> > > > > by
> > > > >> > > > > > a
> > > > >> > > > > > > > > system
> > > > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > As for the Java API, personally, I find it
> > > strange
> > > > >> to
> > > > >> > add
> > > > >> > > > > > 'force'
> > > > >> > > > > > > > or
> > > > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> > > unlikely
> > > > >> that
> > > > >> > > such
> > > > >> > > > > an
> > > > >> > > > > > > > > invocation
> > > > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > > > during
> > > > >> the
> > > > >> > > > > testing
> > > > >> > > > > > > > > phase and
> > > > >> > > > > > > > > > > > developers will end up hard-coding 'true'
> as a
> > > > flag
> > > > >> > > > anyways.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > --
> > > > >> > > > > > > > > Best regards,
> > > > >> > > > > > > > > Ivan Pavlukhin
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > --
> > > > >> > > > > >
> > > > >> > > > > > Best regards,
> > > > >> > > > > > Alexei Scherbakov
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Nikolay Izhikov <ni...@apache.org>.
> To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag
Vladimir, please, go for it.
пт, 7 февр. 2020 г., 11:19 Vladimir Steshin <vl...@gmail.com>:
> Ok. Then we are at the beginning. To prevent unexpected data loss on
> deactivation CLI and JMX should require 'force' flag. If there are no extra
> proposals I'm going to finish the pull request.
>
> чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com
> >:
>
> > Vladimir,
> >
> > IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> > clearing.
> > It allows allocated memory reuse on re-activation to avoid OS specific
> > issues if allocated heap is rather large.
> >
> > You are right to create separate ticket for implementing required
> behavior.
> >
> > чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
> >
> > > Or, is flag [1] is actual only for persistence mode? Related ticket [2]
> > is
> > > completely about disabled persistence.
> > > If previous assumption is right, I think, we can involve flag [1] in
> > ticket
> > > [2].
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> > >
> > > > Denis, Alexei,
> > > >
> > > > Regarding usage of flag
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > [1]
> > > >
> > > > When enabled, I think the following test should work. But it fails.
> > > >
> > > > //----------------------------------------------------------------
> > > > @Test
> > > > public void testDataPresent() throws Exception {
> > > > System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE,
> "true");
> > > >
> > > > IgniteEx i = startGrid(0);
> > > >
> > > > assertFalse(
> > > >
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > > > .isPersistenceEnabled() );
> > > >
> > > > String name = "non-persistent-cache";
> > > >
> > > > i.createCache(name).put(1L, 1L);
> > > >
> > > > assertEquals(1L, i.cache(name).get(1L));
> > > >
> > > > i.cluster().state(ClusterState.INACTIVE);
> > > > i.cluster().state(ClusterState.ACTIVE);
> > > >
> > > > assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> > here!
> > > > }
> > > > //----------------------------------------------------------------
> > > >
> > > >
> > > > Several notes:
> > > >
> > > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > > > is true
> > > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > > > is called with shutdown == false
> > > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > > > is called with deallocate == false
> > > >
> > > > But the cache from the test still has zero size after reactivation.
> > > >
> > > > Is flag [1] disabled by default because it is not implemented /
> doesn't
> > > > work? Do we need to skip it in current ticket and rise new one?
> > > >
> > > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > > >
> > > >> I believe there might be a consistency-related reason why this flag
> > was
> > > >> disabled by default for caches that store data in Ignite native
> > > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some
> light
> > on
> > > >> this.
> > > >>
> > > >> As for the memory-only caches or caches backed up by a CacheStore
> such
> > > as
> > > >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> > > we'll
> > > >> eliminate the need to load the data back into the cluster which can
> > be a
> > > >> time-consuming operation depending on the data volume.
> > > >>
> > > >>
> > > >> -
> > > >> Denis
> > > >>
> > > >>
> > > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <
> vladsz83@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> > There
> > > >> > should be a reason for that. Any data consistency issues when node
> > > gets
> > > >> > activated anew? We may use both solutions because the flag can be
> > > >> switched
> > > >> > off again.
> > > >> >
> > > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > > >> >
> > > >> > > Hi Vladimir,
> > > >> > >
> > > >> > > Yes, I'm suggesting us to enable this flag by
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > default instead of introducing --force flag and showing any
> > > warnings.
> > > >> > >
> > > >> > > -
> > > >> > > Denis
> > > >> > >
> > > >> > >
> > > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> > vladsz83@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hello all.
> > > >> > > >
> > > >> > > > Denis, which changes exactly? In current implementation of
> > ticket
> > > >> [2]
> > > >> > > flag
> > > >> > > > [1] is checked before requiring --force flag and showing any
> > > >> warnings.
> > > >> > Do
> > > >> > > > we need to set reuse-memory-on-deactivate to true by default?
> > > >> > > >
> > > >> > > > [1]
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > >
> > > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >> > > >
> > > >> > > >
> > > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > > >> > > >
> > > >> > > > > That's the best solution for this scenario. Should we
> readjust
> > > the
> > > >> > > > already
> > > >> > > > > created ticket [1] suggesting to implement the changes of
> Alex
> > > >> > > Scherbakov
> > > >> > > > > instead?
> > > >> > > > >
> > > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >> > > > >
> > > >> > > > > -
> > > >> > > > > Denis
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > >> > > > >
> > > >> > > > > > Folks,
> > > >> > > > > >
> > > >> > > > > > For a long time we have a flag [1]
> > > >> > > > > >
> > > >> > > > > > It does almost what we want here.
> > > >> > > > > >
> > > >> > > > > > I suggest to make this behavior default and rework it to
> > keep
> > > >> data
> > > >> > in
> > > >> > > > > > memory as well (we already have special "recovery" mode
> for
> > > >> > caches).
> > > >> > > > > >
> > > >> > > > > > [1]
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > >> > > > > alexey.goncharuk@gmail.com
> > > >> > > > > > >:
> > > >> > > > > >
> > > >> > > > > > > I do not mind making this change if we explicitly and
> > > clearly
> > > >> > > define
> > > >> > > > > what
> > > >> > > > > > > 'new inactive state' means. What should happen if a
> > > partition
> > > >> is
> > > >> > > lost
> > > >> > > > > in
> > > >> > > > > > > inactive state? What if such node joins back the cluster
> > > >> after?
> > > >> > > Etc.
> > > >> > > > > > >
> > > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> > dmagda@apache.org
> > > >:
> > > >> > > > > > >
> > > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > >> > > activation/deactivation
> > > >> > > > > was
> > > >> > > > > > > > created for the baseline topology designed for cases
> > with
> > > >> > native
> > > >> > > > > > > > persistence. My thinking was that the mechanism
> itended
> > to
> > > >> > > prevent
> > > >> > > > > data
> > > >> > > > > > > > inconsistencies while nodes with data on the disk will
> > be
> > > in
> > > >> > the
> > > >> > > > > > process
> > > >> > > > > > > of
> > > >> > > > > > > > joining the cluster.
> > > >> > > > > > > >
> > > >> > > > > > > > Artem, could you please update the docs bringing this
> to
> > > the
> > > >> > > > > attention
> > > >> > > > > > of
> > > >> > > > > > > > the user community?
> > > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > >> > > > > > > >
> > > >> > > > > > > > AG, what if we don't purge data from memory at least
> for
> > > the
> > > >> > > caches
> > > >> > > > > not
> > > >> > > > > > > > backed by the native persistence? Is this a big deal?
> We
> > > can
> > > >> > > > > certainly
> > > >> > > > > > > put
> > > >> > > > > > > > this off by my guts feel we'll return to this question
> > > >> sooner
> > > >> > or
> > > >> > > > > later.
> > > >> > > > > > > >
> > > >> > > > > > > > -
> > > >> > > > > > > > Denis
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > >> > > > vololo100@gmail.com>
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > For me it looks like some coincidence effect. I
> > > understand
> > > >> > that
> > > >> > > > we
> > > >> > > > > > get
> > > >> > > > > > > > > such behavior because deactivation works the same
> way
> > as
> > > >> for
> > > >> > > > > > > > > persistent caches. Was cluster
> activation/deactivation
> > > >> > designed
> > > >> > > > and
> > > >> > > > > > > > > described for in-memory caches? Current behavior
> > sounds
> > > >> for
> > > >> > me
> > > >> > > a
> > > >> > > > as
> > > >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> > > >> purged
> > > >> > all
> > > >> > > > > their
> > > >> > > > > > > > > data.
> > > >> > > > > > > > >
> > > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > >> > > > > > > > alexey.goncharuk@gmail.com
> > > >> > > > > > > > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Because originally the sole purpose of
> deactivation
> > > was
> > > >> > > > resource
> > > >> > > > > > > > > > deallocation.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > > >> > dmagda@apache.org
> > > >> > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > > Such a revelation for me that data is purged
> from
> > > RAM
> > > >> if
> > > >> > > > > someone
> > > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember
> why
> > > we
> > > >> > > decided
> > > >> > > > > to
> > > >> > > > > > > > > implement
> > > >> > > > > > > > > > > it this way initially?
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > -
> > > >> > > > > > > > > > > Denis
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey
> Goncharuk <
> > > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > >> > > > > > > > > > > wrote:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > > I agree on CLI and JMX because those
> interfaces
> > > can
> > > >> be
> > > >> > > used
> > > >> > > > > by
> > > >> > > > > > a
> > > >> > > > > > > > > system
> > > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > As for the Java API, personally, I find it
> > strange
> > > >> to
> > > >> > add
> > > >> > > > > > 'force'
> > > >> > > > > > > > or
> > > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> > unlikely
> > > >> that
> > > >> > > such
> > > >> > > > > an
> > > >> > > > > > > > > invocation
> > > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > > during
> > > >> the
> > > >> > > > > testing
> > > >> > > > > > > > > phase and
> > > >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> > > flag
> > > >> > > > anyways.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > --
> > > >> > > > > > > > > Best regards,
> > > >> > > > > > > > > Ivan Pavlukhin
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > --
> > > >> > > > > >
> > > >> > > > > > Best regards,
> > > >> > > > > > Alexei Scherbakov
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Ok. Then we are at the beginning. To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag. If there are no extra
proposals I'm going to finish the pull request.
чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:
> Vladimir,
>
> IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> clearing.
> It allows allocated memory reuse on re-activation to avoid OS specific
> issues if allocated heap is rather large.
>
> You are right to create separate ticket for implementing required behavior.
>
> чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
>
> > Or, is flag [1] is actual only for persistence mode? Related ticket [2]
> is
> > completely about disabled persistence.
> > If previous assumption is right, I think, we can involve flag [1] in
> ticket
> > [2].
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> >
> > > Denis, Alexei,
> > >
> > > Regarding usage of flag
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > [1]
> > >
> > > When enabled, I think the following test should work. But it fails.
> > >
> > > //----------------------------------------------------------------
> > > @Test
> > > public void testDataPresent() throws Exception {
> > > System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
> > >
> > > IgniteEx i = startGrid(0);
> > >
> > > assertFalse(
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > > .isPersistenceEnabled() );
> > >
> > > String name = "non-persistent-cache";
> > >
> > > i.createCache(name).put(1L, 1L);
> > >
> > > assertEquals(1L, i.cache(name).get(1L));
> > >
> > > i.cluster().state(ClusterState.INACTIVE);
> > > i.cluster().state(ClusterState.ACTIVE);
> > >
> > > assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> here!
> > > }
> > > //----------------------------------------------------------------
> > >
> > >
> > > Several notes:
> > >
> > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > > is true
> > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > > is called with shutdown == false
> > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > > is called with deallocate == false
> > >
> > > But the cache from the test still has zero size after reactivation.
> > >
> > > Is flag [1] disabled by default because it is not implemented / doesn't
> > > work? Do we need to skip it in current ticket and rise new one?
> > >
> > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > >
> > >> I believe there might be a consistency-related reason why this flag
> was
> > >> disabled by default for caches that store data in Ignite native
> > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light
> on
> > >> this.
> > >>
> > >> As for the memory-only caches or caches backed up by a CacheStore such
> > as
> > >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> > we'll
> > >> eliminate the need to load the data back into the cluster which can
> be a
> > >> time-consuming operation depending on the data volume.
> > >>
> > >>
> > >> -
> > >> Denis
> > >>
> > >>
> > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> > >> wrote:
> > >>
> > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> There
> > >> > should be a reason for that. Any data consistency issues when node
> > gets
> > >> > activated anew? We may use both solutions because the flag can be
> > >> switched
> > >> > off again.
> > >> >
> > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > >> >
> > >> > > Hi Vladimir,
> > >> > >
> > >> > > Yes, I'm suggesting us to enable this flag by
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > default instead of introducing --force flag and showing any
> > warnings.
> > >> > >
> > >> > > -
> > >> > > Denis
> > >> > >
> > >> > >
> > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> vladsz83@gmail.com
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Hello all.
> > >> > > >
> > >> > > > Denis, which changes exactly? In current implementation of
> ticket
> > >> [2]
> > >> > > flag
> > >> > > > [1] is checked before requiring --force flag and showing any
> > >> warnings.
> > >> > Do
> > >> > > > we need to set reuse-memory-on-deactivate to true by default?
> > >> > > >
> > >> > > > [1]
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > >
> > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >> > > >
> > >> > > >
> > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > >> > > >
> > >> > > > > That's the best solution for this scenario. Should we readjust
> > the
> > >> > > > already
> > >> > > > > created ticket [1] suggesting to implement the changes of Alex
> > >> > > Scherbakov
> > >> > > > > instead?
> > >> > > > >
> > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > >> > > > >
> > >> > > > > -
> > >> > > > > Denis
> > >> > > > >
> > >> > > > >
> > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > >> > > > >
> > >> > > > > > Folks,
> > >> > > > > >
> > >> > > > > > For a long time we have a flag [1]
> > >> > > > > >
> > >> > > > > > It does almost what we want here.
> > >> > > > > >
> > >> > > > > > I suggest to make this behavior default and rework it to
> keep
> > >> data
> > >> > in
> > >> > > > > > memory as well (we already have special "recovery" mode for
> > >> > caches).
> > >> > > > > >
> > >> > > > > > [1]
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > >> > > > > alexey.goncharuk@gmail.com
> > >> > > > > > >:
> > >> > > > > >
> > >> > > > > > > I do not mind making this change if we explicitly and
> > clearly
> > >> > > define
> > >> > > > > what
> > >> > > > > > > 'new inactive state' means. What should happen if a
> > partition
> > >> is
> > >> > > lost
> > >> > > > > in
> > >> > > > > > > inactive state? What if such node joins back the cluster
> > >> after?
> > >> > > Etc.
> > >> > > > > > >
> > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> dmagda@apache.org
> > >:
> > >> > > > > > >
> > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > >> > > activation/deactivation
> > >> > > > > was
> > >> > > > > > > > created for the baseline topology designed for cases
> with
> > >> > native
> > >> > > > > > > > persistence. My thinking was that the mechanism itended
> to
> > >> > > prevent
> > >> > > > > data
> > >> > > > > > > > inconsistencies while nodes with data on the disk will
> be
> > in
> > >> > the
> > >> > > > > > process
> > >> > > > > > > of
> > >> > > > > > > > joining the cluster.
> > >> > > > > > > >
> > >> > > > > > > > Artem, could you please update the docs bringing this to
> > the
> > >> > > > > attention
> > >> > > > > > of
> > >> > > > > > > > the user community?
> > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > >> > > > > > > >
> > >> > > > > > > > AG, what if we don't purge data from memory at least for
> > the
> > >> > > caches
> > >> > > > > not
> > >> > > > > > > > backed by the native persistence? Is this a big deal? We
> > can
> > >> > > > > certainly
> > >> > > > > > > put
> > >> > > > > > > > this off by my guts feel we'll return to this question
> > >> sooner
> > >> > or
> > >> > > > > later.
> > >> > > > > > > >
> > >> > > > > > > > -
> > >> > > > > > > > Denis
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > >> > > > vololo100@gmail.com>
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > For me it looks like some coincidence effect. I
> > understand
> > >> > that
> > >> > > > we
> > >> > > > > > get
> > >> > > > > > > > > such behavior because deactivation works the same way
> as
> > >> for
> > >> > > > > > > > > persistent caches. Was cluster activation/deactivation
> > >> > designed
> > >> > > > and
> > >> > > > > > > > > described for in-memory caches? Current behavior
> sounds
> > >> for
> > >> > me
> > >> > > a
> > >> > > > as
> > >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> > >> purged
> > >> > all
> > >> > > > > their
> > >> > > > > > > > > data.
> > >> > > > > > > > >
> > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > >> > > > > > > > alexey.goncharuk@gmail.com
> > >> > > > > > > > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > > Because originally the sole purpose of deactivation
> > was
> > >> > > > resource
> > >> > > > > > > > > > deallocation.
> > >> > > > > > > > > >
> > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > >> > dmagda@apache.org
> > >> > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Such a revelation for me that data is purged from
> > RAM
> > >> if
> > >> > > > > someone
> > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember why
> > we
> > >> > > decided
> > >> > > > > to
> > >> > > > > > > > > implement
> > >> > > > > > > > > > > it this way initially?
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > -
> > >> > > > > > > > > > > Denis
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > >> > > > > > > > > > > wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > > I agree on CLI and JMX because those interfaces
> > can
> > >> be
> > >> > > used
> > >> > > > > by
> > >> > > > > > a
> > >> > > > > > > > > system
> > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > As for the Java API, personally, I find it
> strange
> > >> to
> > >> > add
> > >> > > > > > 'force'
> > >> > > > > > > > or
> > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> unlikely
> > >> that
> > >> > > such
> > >> > > > > an
> > >> > > > > > > > > invocation
> > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > during
> > >> the
> > >> > > > > testing
> > >> > > > > > > > > phase and
> > >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> > flag
> > >> > > > anyways.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > --
> > >> > > > > > > > > Best regards,
> > >> > > > > > > > > Ivan Pavlukhin
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > >
> > >> > > > > > Best regards,
> > >> > > > > > Alexei Scherbakov
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Alexei Scherbakov <al...@gmail.com>.
Vladimir,
IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
clearing.
It allows allocated memory reuse on re-activation to avoid OS specific
issues if allocated heap is rather large.
You are right to create separate ticket for implementing required behavior.
чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
> Or, is flag [1] is actual only for persistence mode? Related ticket [2] is
> completely about disabled persistence.
> If previous assumption is right, I think, we can involve flag [1] in ticket
> [2].
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> [2] https://issues.apache.org/jira/browse/IGNITE-12614
>
> чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
>
> > Denis, Alexei,
> >
> > Regarding usage of flag
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > [1]
> >
> > When enabled, I think the following test should work. But it fails.
> >
> > //----------------------------------------------------------------
> > @Test
> > public void testDataPresent() throws Exception {
> > System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
> >
> > IgniteEx i = startGrid(0);
> >
> > assertFalse(
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > .isPersistenceEnabled() );
> >
> > String name = "non-persistent-cache";
> >
> > i.createCache(name).put(1L, 1L);
> >
> > assertEquals(1L, i.cache(name).get(1L));
> >
> > i.cluster().state(ClusterState.INACTIVE);
> > i.cluster().state(ClusterState.ACTIVE);
> >
> > assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
> > }
> > //----------------------------------------------------------------
> >
> >
> > Several notes:
> >
> > - IgniteCacheDatabaseSharedManager#reuseMemory
> > is true
> > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > is called with shutdown == false
> > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > is called with deallocate == false
> >
> > But the cache from the test still has zero size after reactivation.
> >
> > Is flag [1] disabled by default because it is not implemented / doesn't
> > work? Do we need to skip it in current ticket and rise new one?
> >
> > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> >
> >> I believe there might be a consistency-related reason why this flag was
> >> disabled by default for caches that store data in Ignite native
> >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
> >> this.
> >>
> >> As for the memory-only caches or caches backed up by a CacheStore such
> as
> >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> we'll
> >> eliminate the need to load the data back into the cluster which can be a
> >> time-consuming operation depending on the data volume.
> >>
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> >> wrote:
> >>
> >> > Denis, but why reuse-data-on-deactivate was disabled by default? There
> >> > should be a reason for that. Any data consistency issues when node
> gets
> >> > activated anew? We may use both solutions because the flag can be
> >> switched
> >> > off again.
> >> >
> >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> >> >
> >> > > Hi Vladimir,
> >> > >
> >> > > Yes, I'm suggesting us to enable this flag by
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > default instead of introducing --force flag and showing any
> warnings.
> >> > >
> >> > > -
> >> > > Denis
> >> > >
> >> > >
> >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vladsz83@gmail.com
> >
> >> > > wrote:
> >> > >
> >> > > > Hello all.
> >> > > >
> >> > > > Denis, which changes exactly? In current implementation of ticket
> >> [2]
> >> > > flag
> >> > > > [1] is checked before requiring --force flag and showing any
> >> warnings.
> >> > Do
> >> > > > we need to set reuse-memory-on-deactivate to true by default?
> >> > > >
> >> > > > [1]
> >> > > >
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > >
> >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >> > > >
> >> > > >
> >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> >> > > >
> >> > > > > That's the best solution for this scenario. Should we readjust
> the
> >> > > > already
> >> > > > > created ticket [1] suggesting to implement the changes of Alex
> >> > > Scherbakov
> >> > > > > instead?
> >> > > > >
> >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >> > > > >
> >> > > > > -
> >> > > > > Denis
> >> > > > >
> >> > > > >
> >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> >> > > > > alexey.scherbakoff@gmail.com> wrote:
> >> > > > >
> >> > > > > > Folks,
> >> > > > > >
> >> > > > > > For a long time we have a flag [1]
> >> > > > > >
> >> > > > > > It does almost what we want here.
> >> > > > > >
> >> > > > > > I suggest to make this behavior default and rework it to keep
> >> data
> >> > in
> >> > > > > > memory as well (we already have special "recovery" mode for
> >> > caches).
> >> > > > > >
> >> > > > > > [1]
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> >> > > > > alexey.goncharuk@gmail.com
> >> > > > > > >:
> >> > > > > >
> >> > > > > > > I do not mind making this change if we explicitly and
> clearly
> >> > > define
> >> > > > > what
> >> > > > > > > 'new inactive state' means. What should happen if a
> partition
> >> is
> >> > > lost
> >> > > > > in
> >> > > > > > > inactive state? What if such node joins back the cluster
> >> after?
> >> > > Etc.
> >> > > > > > >
> >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dmagda@apache.org
> >:
> >> > > > > > >
> >> > > > > > > > Back up Ivan's opinion here. Initially, the
> >> > > activation/deactivation
> >> > > > > was
> >> > > > > > > > created for the baseline topology designed for cases with
> >> > native
> >> > > > > > > > persistence. My thinking was that the mechanism itended to
> >> > > prevent
> >> > > > > data
> >> > > > > > > > inconsistencies while nodes with data on the disk will be
> in
> >> > the
> >> > > > > > process
> >> > > > > > > of
> >> > > > > > > > joining the cluster.
> >> > > > > > > >
> >> > > > > > > > Artem, could you please update the docs bringing this to
> the
> >> > > > > attention
> >> > > > > > of
> >> > > > > > > > the user community?
> >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> >> > > > > > > >
> >> > > > > > > > AG, what if we don't purge data from memory at least for
> the
> >> > > caches
> >> > > > > not
> >> > > > > > > > backed by the native persistence? Is this a big deal? We
> can
> >> > > > > certainly
> >> > > > > > > put
> >> > > > > > > > this off by my guts feel we'll return to this question
> >> sooner
> >> > or
> >> > > > > later.
> >> > > > > > > >
> >> > > > > > > > -
> >> > > > > > > > Denis
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> >> > > > vololo100@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > For me it looks like some coincidence effect. I
> understand
> >> > that
> >> > > > we
> >> > > > > > get
> >> > > > > > > > > such behavior because deactivation works the same way as
> >> for
> >> > > > > > > > > persistent caches. Was cluster activation/deactivation
> >> > designed
> >> > > > and
> >> > > > > > > > > described for in-memory caches? Current behavior sounds
> >> for
> >> > me
> >> > > a
> >> > > > as
> >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> >> purged
> >> > all
> >> > > > > their
> >> > > > > > > > > data.
> >> > > > > > > > >
> >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> >> > > > > > > > alexey.goncharuk@gmail.com
> >> > > > > > > > > >:
> >> > > > > > > > > >
> >> > > > > > > > > > Because originally the sole purpose of deactivation
> was
> >> > > > resource
> >> > > > > > > > > > deallocation.
> >> > > > > > > > > >
> >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> >> > dmagda@apache.org
> >> > > >:
> >> > > > > > > > > >
> >> > > > > > > > > > > Such a revelation for me that data is purged from
> RAM
> >> if
> >> > > > > someone
> >> > > > > > > > > > > deactivates the cluster. Alex, do you remember why
> we
> >> > > decided
> >> > > > > to
> >> > > > > > > > > implement
> >> > > > > > > > > > > it this way initially?
> >> > > > > > > > > > >
> >> > > > > > > > > > > -
> >> > > > > > > > > > > Denis
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > I agree on CLI and JMX because those interfaces
> can
> >> be
> >> > > used
> >> > > > > by
> >> > > > > > a
> >> > > > > > > > > system
> >> > > > > > > > > > > > administrator and can be invoked by mistake.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > As for the Java API, personally, I find it strange
> >> to
> >> > add
> >> > > > > > 'force'
> >> > > > > > > > or
> >> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
> >> that
> >> > > such
> >> > > > > an
> >> > > > > > > > > invocation
> >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> during
> >> the
> >> > > > > testing
> >> > > > > > > > > phase and
> >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> flag
> >> > > > anyways.
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Best regards,
> >> > > > > > > > > Ivan Pavlukhin
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > >
> >> > > > > > Best regards,
> >> > > > > > Alexei Scherbakov
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
--
Best regards,
Alexei Scherbakov
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Or, is flag [1] is actual only for persistence mode? Related ticket [2] is
completely about disabled persistence.
If previous assumption is right, I think, we can involve flag [1] in ticket
[2].
[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[2] https://issues.apache.org/jira/browse/IGNITE-12614
чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> Denis, Alexei,
>
> Regarding usage of flag
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> [1]
>
> When enabled, I think the following test should work. But it fails.
>
> //----------------------------------------------------------------
> @Test
> public void testDataPresent() throws Exception {
> System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
>
> IgniteEx i = startGrid(0);
>
> assertFalse(
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> .isPersistenceEnabled() );
>
> String name = "non-persistent-cache";
>
> i.createCache(name).put(1L, 1L);
>
> assertEquals(1L, i.cache(name).get(1L));
>
> i.cluster().state(ClusterState.INACTIVE);
> i.cluster().state(ClusterState.ACTIVE);
>
> assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
> }
> //----------------------------------------------------------------
>
>
> Several notes:
>
> - IgniteCacheDatabaseSharedManager#reuseMemory
> is true
> - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> is called with shutdown == false
> - PageMemoryNoStoreImpl#stop(booleam deallocate)
> is called with deallocate == false
>
> But the cache from the test still has zero size after reactivation.
>
> Is flag [1] disabled by default because it is not implemented / doesn't
> work? Do we need to skip it in current ticket and rise new one?
>
> ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
>
>> I believe there might be a consistency-related reason why this flag was
>> disabled by default for caches that store data in Ignite native
>> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
>> this.
>>
>> As for the memory-only caches or caches backed up by a CacheStore such as
>> an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
>> eliminate the need to load the data back into the cluster which can be a
>> time-consuming operation depending on the data volume.
>>
>>
>> -
>> Denis
>>
>>
>> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
>> wrote:
>>
>> > Denis, but why reuse-data-on-deactivate was disabled by default? There
>> > should be a reason for that. Any data consistency issues when node gets
>> > activated anew? We may use both solutions because the flag can be
>> switched
>> > off again.
>> >
>> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
>> >
>> > > Hi Vladimir,
>> > >
>> > > Yes, I'm suggesting us to enable this flag by
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > default instead of introducing --force flag and showing any warnings.
>> > >
>> > > -
>> > > Denis
>> > >
>> > >
>> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello all.
>> > > >
>> > > > Denis, which changes exactly? In current implementation of ticket
>> [2]
>> > > flag
>> > > > [1] is checked before requiring --force flag and showing any
>> warnings.
>> > Do
>> > > > we need to set reuse-memory-on-deactivate to true by default?
>> > > >
>> > > > [1]
>> > > >
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > >
>> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
>> > > >
>> > > >
>> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
>> > > >
>> > > > > That's the best solution for this scenario. Should we readjust the
>> > > > already
>> > > > > created ticket [1] suggesting to implement the changes of Alex
>> > > Scherbakov
>> > > > > instead?
>> > > > >
>> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
>> > > > > alexey.scherbakoff@gmail.com> wrote:
>> > > > >
>> > > > > > Folks,
>> > > > > >
>> > > > > > For a long time we have a flag [1]
>> > > > > >
>> > > > > > It does almost what we want here.
>> > > > > >
>> > > > > > I suggest to make this behavior default and rework it to keep
>> data
>> > in
>> > > > > > memory as well (we already have special "recovery" mode for
>> > caches).
>> > > > > >
>> > > > > > [1]
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
>> > > > > alexey.goncharuk@gmail.com
>> > > > > > >:
>> > > > > >
>> > > > > > > I do not mind making this change if we explicitly and clearly
>> > > define
>> > > > > what
>> > > > > > > 'new inactive state' means. What should happen if a partition
>> is
>> > > lost
>> > > > > in
>> > > > > > > inactive state? What if such node joins back the cluster
>> after?
>> > > Etc.
>> > > > > > >
>> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>> > > > > > >
>> > > > > > > > Back up Ivan's opinion here. Initially, the
>> > > activation/deactivation
>> > > > > was
>> > > > > > > > created for the baseline topology designed for cases with
>> > native
>> > > > > > > > persistence. My thinking was that the mechanism itended to
>> > > prevent
>> > > > > data
>> > > > > > > > inconsistencies while nodes with data on the disk will be in
>> > the
>> > > > > > process
>> > > > > > > of
>> > > > > > > > joining the cluster.
>> > > > > > > >
>> > > > > > > > Artem, could you please update the docs bringing this to the
>> > > > > attention
>> > > > > > of
>> > > > > > > > the user community?
>> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
>> > > > > > > >
>> > > > > > > > AG, what if we don't purge data from memory at least for the
>> > > caches
>> > > > > not
>> > > > > > > > backed by the native persistence? Is this a big deal? We can
>> > > > > certainly
>> > > > > > > put
>> > > > > > > > this off by my guts feel we'll return to this question
>> sooner
>> > or
>> > > > > later.
>> > > > > > > >
>> > > > > > > > -
>> > > > > > > > Denis
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
>> > > > vololo100@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > For me it looks like some coincidence effect. I understand
>> > that
>> > > > we
>> > > > > > get
>> > > > > > > > > such behavior because deactivation works the same way as
>> for
>> > > > > > > > > persistent caches. Was cluster activation/deactivation
>> > designed
>> > > > and
>> > > > > > > > > described for in-memory caches? Current behavior sounds
>> for
>> > me
>> > > a
>> > > > as
>> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
>> purged
>> > all
>> > > > > their
>> > > > > > > > > data.
>> > > > > > > > >
>> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
>> > > > > > > > alexey.goncharuk@gmail.com
>> > > > > > > > > >:
>> > > > > > > > > >
>> > > > > > > > > > Because originally the sole purpose of deactivation was
>> > > > resource
>> > > > > > > > > > deallocation.
>> > > > > > > > > >
>> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
>> > dmagda@apache.org
>> > > >:
>> > > > > > > > > >
>> > > > > > > > > > > Such a revelation for me that data is purged from RAM
>> if
>> > > > > someone
>> > > > > > > > > > > deactivates the cluster. Alex, do you remember why we
>> > > decided
>> > > > > to
>> > > > > > > > > implement
>> > > > > > > > > > > it this way initially?
>> > > > > > > > > > >
>> > > > > > > > > > > -
>> > > > > > > > > > > Denis
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
>> > > > > > > > > > > alexey.goncharuk@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > I agree on CLI and JMX because those interfaces can
>> be
>> > > used
>> > > > > by
>> > > > > > a
>> > > > > > > > > system
>> > > > > > > > > > > > administrator and can be invoked by mistake.
>> > > > > > > > > > > >
>> > > > > > > > > > > > As for the Java API, personally, I find it strange
>> to
>> > add
>> > > > > > 'force'
>> > > > > > > > or
>> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
>> that
>> > > such
>> > > > > an
>> > > > > > > > > invocation
>> > > > > > > > > > > > is done by mistake. Such mistakes are caught during
>> the
>> > > > > testing
>> > > > > > > > > phase and
>> > > > > > > > > > > > developers will end up hard-coding 'true' as a flag
>> > > > anyways.
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Best regards,
>> > > > > > > > > Ivan Pavlukhin
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > >
>> > > > > > Best regards,
>> > > > > > Alexei Scherbakov
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Denis, Alexei,
Regarding usage of flag
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[1]
When enabled, I think the following test should work. But it fails.
//----------------------------------------------------------------
@Test
public void testDataPresent() throws Exception {
System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
IgniteEx i = startGrid(0);
assertFalse(
i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
.isPersistenceEnabled() );
String name = "non-persistent-cache";
i.createCache(name).put(1L, 1L);
assertEquals(1L, i.cache(name).get(1L));
i.cluster().state(ClusterState.INACTIVE);
i.cluster().state(ClusterState.ACTIVE);
assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
}
//----------------------------------------------------------------
Several notes:
- IgniteCacheDatabaseSharedManager#reuseMemory
is true
- IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
is called with shutdown == false
- PageMemoryNoStoreImpl#stop(booleam deallocate)
is called with deallocate == false
But the cache from the test still has zero size after reactivation.
Is flag [1] disabled by default because it is not implemented / doesn't
work? Do we need to skip it in current ticket and rise new one?
ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> I believe there might be a consistency-related reason why this flag was
> disabled by default for caches that store data in Ignite native
> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
> this.
>
> As for the memory-only caches or caches backed up by a CacheStore such as
> an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
> eliminate the need to load the data back into the cluster which can be a
> time-consuming operation depending on the data volume.
>
>
> -
> Denis
>
>
> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> wrote:
>
> > Denis, but why reuse-data-on-deactivate was disabled by default? There
> > should be a reason for that. Any data consistency issues when node gets
> > activated anew? We may use both solutions because the flag can be
> switched
> > off again.
> >
> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> >
> > > Hi Vladimir,
> > >
> > > Yes, I'm suggesting us to enable this flag by
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > default instead of introducing --force flag and showing any warnings.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> > > wrote:
> > >
> > > > Hello all.
> > > >
> > > > Denis, which changes exactly? In current implementation of ticket [2]
> > > flag
> > > > [1] is checked before requiring --force flag and showing any
> warnings.
> > Do
> > > > we need to set reuse-memory-on-deactivate to true by default?
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > >
> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > > >
> > > > > That's the best solution for this scenario. Should we readjust the
> > > > already
> > > > > created ticket [1] suggesting to implement the changes of Alex
> > > Scherbakov
> > > > > instead?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > For a long time we have a flag [1]
> > > > > >
> > > > > > It does almost what we want here.
> > > > > >
> > > > > > I suggest to make this behavior default and rework it to keep
> data
> > in
> > > > > > memory as well (we already have special "recovery" mode for
> > caches).
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > > >
> > > > > >
> > > > > >
> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > > >:
> > > > > >
> > > > > > > I do not mind making this change if we explicitly and clearly
> > > define
> > > > > what
> > > > > > > 'new inactive state' means. What should happen if a partition
> is
> > > lost
> > > > > in
> > > > > > > inactive state? What if such node joins back the cluster after?
> > > Etc.
> > > > > > >
> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > > > >
> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > activation/deactivation
> > > > > was
> > > > > > > > created for the baseline topology designed for cases with
> > native
> > > > > > > > persistence. My thinking was that the mechanism itended to
> > > prevent
> > > > > data
> > > > > > > > inconsistencies while nodes with data on the disk will be in
> > the
> > > > > > process
> > > > > > > of
> > > > > > > > joining the cluster.
> > > > > > > >
> > > > > > > > Artem, could you please update the docs bringing this to the
> > > > > attention
> > > > > > of
> > > > > > > > the user community?
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > > > >
> > > > > > > > AG, what if we don't purge data from memory at least for the
> > > caches
> > > > > not
> > > > > > > > backed by the native persistence? Is this a big deal? We can
> > > > > certainly
> > > > > > > put
> > > > > > > > this off by my guts feel we'll return to this question sooner
> > or
> > > > > later.
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > > vololo100@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > For me it looks like some coincidence effect. I understand
> > that
> > > > we
> > > > > > get
> > > > > > > > > such behavior because deactivation works the same way as
> for
> > > > > > > > > persistent caches. Was cluster activation/deactivation
> > designed
> > > > and
> > > > > > > > > described for in-memory caches? Current behavior sounds for
> > me
> > > a
> > > > as
> > > > > > > > > big risk. I expect a lot of upset users unexpectedly purged
> > all
> > > > > their
> > > > > > > > > data.
> > > > > > > > >
> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > Because originally the sole purpose of deactivation was
> > > > resource
> > > > > > > > > > deallocation.
> > > > > > > > > >
> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > dmagda@apache.org
> > > >:
> > > > > > > > > >
> > > > > > > > > > > Such a revelation for me that data is purged from RAM
> if
> > > > > someone
> > > > > > > > > > > deactivates the cluster. Alex, do you remember why we
> > > decided
> > > > > to
> > > > > > > > > implement
> > > > > > > > > > > it this way initially?
> > > > > > > > > > >
> > > > > > > > > > > -
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I agree on CLI and JMX because those interfaces can
> be
> > > used
> > > > > by
> > > > > > a
> > > > > > > > > system
> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > > > >
> > > > > > > > > > > > As for the Java API, personally, I find it strange to
> > add
> > > > > > 'force'
> > > > > > > > or
> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
> that
> > > such
> > > > > an
> > > > > > > > > invocation
> > > > > > > > > > > > is done by mistake. Such mistakes are caught during
> the
> > > > > testing
> > > > > > > > > phase and
> > > > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > > > anyways.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Ivan Pavlukhin
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best regards,
> > > > > > Alexei Scherbakov
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Denis Magda <dm...@apache.org>.
I believe there might be a consistency-related reason why this flag was
disabled by default for caches that store data in Ignite native
persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
this.
As for the memory-only caches or caches backed up by a CacheStore such as
an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
eliminate the need to load the data back into the cluster which can be a
time-consuming operation depending on the data volume.
-
Denis
On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com> wrote:
> Denis, but why reuse-data-on-deactivate was disabled by default? There
> should be a reason for that. Any data consistency issues when node gets
> activated anew? We may use both solutions because the flag can be switched
> off again.
>
> ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
>
> > Hi Vladimir,
> >
> > Yes, I'm suggesting us to enable this flag by
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > default instead of introducing --force flag and showing any warnings.
> >
> > -
> > Denis
> >
> >
> > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> > wrote:
> >
> > > Hello all.
> > >
> > > Denis, which changes exactly? In current implementation of ticket [2]
> > flag
> > > [1] is checked before requiring --force flag and showing any warnings.
> Do
> > > we need to set reuse-memory-on-deactivate to true by default?
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >
> > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > >
> > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > >
> > > > That's the best solution for this scenario. Should we readjust the
> > > already
> > > > created ticket [1] suggesting to implement the changes of Alex
> > Scherbakov
> > > > instead?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > alexey.scherbakoff@gmail.com> wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > For a long time we have a flag [1]
> > > > >
> > > > > It does almost what we want here.
> > > > >
> > > > > I suggest to make this behavior default and rework it to keep data
> in
> > > > > memory as well (we already have special "recovery" mode for
> caches).
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >
> > > > >
> > > > >
> > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > > >:
> > > > >
> > > > > > I do not mind making this change if we explicitly and clearly
> > define
> > > > what
> > > > > > 'new inactive state' means. What should happen if a partition is
> > lost
> > > > in
> > > > > > inactive state? What if such node joins back the cluster after?
> > Etc.
> > > > > >
> > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > > > Back up Ivan's opinion here. Initially, the
> > activation/deactivation
> > > > was
> > > > > > > created for the baseline topology designed for cases with
> native
> > > > > > > persistence. My thinking was that the mechanism itended to
> > prevent
> > > > data
> > > > > > > inconsistencies while nodes with data on the disk will be in
> the
> > > > > process
> > > > > > of
> > > > > > > joining the cluster.
> > > > > > >
> > > > > > > Artem, could you please update the docs bringing this to the
> > > > attention
> > > > > of
> > > > > > > the user community?
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > > >
> > > > > > > AG, what if we don't purge data from memory at least for the
> > caches
> > > > not
> > > > > > > backed by the native persistence? Is this a big deal? We can
> > > > certainly
> > > > > > put
> > > > > > > this off by my guts feel we'll return to this question sooner
> or
> > > > later.
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > vololo100@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > For me it looks like some coincidence effect. I understand
> that
> > > we
> > > > > get
> > > > > > > > such behavior because deactivation works the same way as for
> > > > > > > > persistent caches. Was cluster activation/deactivation
> designed
> > > and
> > > > > > > > described for in-memory caches? Current behavior sounds for
> me
> > a
> > > as
> > > > > > > > big risk. I expect a lot of upset users unexpectedly purged
> all
> > > > their
> > > > > > > > data.
> > > > > > > >
> > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > >:
> > > > > > > > >
> > > > > > > > > Because originally the sole purpose of deactivation was
> > > resource
> > > > > > > > > deallocation.
> > > > > > > > >
> > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> dmagda@apache.org
> > >:
> > > > > > > > >
> > > > > > > > > > Such a revelation for me that data is purged from RAM if
> > > > someone
> > > > > > > > > > deactivates the cluster. Alex, do you remember why we
> > decided
> > > > to
> > > > > > > > implement
> > > > > > > > > > it this way initially?
> > > > > > > > > >
> > > > > > > > > > -
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I agree on CLI and JMX because those interfaces can be
> > used
> > > > by
> > > > > a
> > > > > > > > system
> > > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > > >
> > > > > > > > > > > As for the Java API, personally, I find it strange to
> add
> > > > > 'force'
> > > > > > > or
> > > > > > > > > > > 'confirm' flags to it because it is very unlikely that
> > such
> > > > an
> > > > > > > > invocation
> > > > > > > > > > > is done by mistake. Such mistakes are caught during the
> > > > testing
> > > > > > > > phase and
> > > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > > anyways.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Ivan Pavlukhin
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > Alexei Scherbakov
> > > > >
> > > >
> > >
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Denis, but why reuse-data-on-deactivate was disabled by default? There
should be a reason for that. Any data consistency issues when node gets
activated anew? We may use both solutions because the flag can be switched
off again.
ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> Hi Vladimir,
>
> Yes, I'm suggesting us to enable this flag by
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> default instead of introducing --force flag and showing any warnings.
>
> -
> Denis
>
>
> On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> wrote:
>
> > Hello all.
> >
> > Denis, which changes exactly? In current implementation of ticket [2]
> flag
> > [1] is checked before requiring --force flag and showing any warnings. Do
> > we need to set reuse-memory-on-deactivate to true by default?
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >
> > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> >
> > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> >
> > > That's the best solution for this scenario. Should we readjust the
> > already
> > > created ticket [1] suggesting to implement the changes of Alex
> Scherbakov
> > > instead?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > alexey.scherbakoff@gmail.com> wrote:
> > >
> > > > Folks,
> > > >
> > > > For a long time we have a flag [1]
> > > >
> > > > It does almost what we want here.
> > > >
> > > > I suggest to make this behavior default and rework it to keep data in
> > > > memory as well (we already have special "recovery" mode for caches).
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >
> > > >
> > > >
> > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >:
> > > >
> > > > > I do not mind making this change if we explicitly and clearly
> define
> > > what
> > > > > 'new inactive state' means. What should happen if a partition is
> lost
> > > in
> > > > > inactive state? What if such node joins back the cluster after?
> Etc.
> > > > >
> > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > >
> > > > > > Back up Ivan's opinion here. Initially, the
> activation/deactivation
> > > was
> > > > > > created for the baseline topology designed for cases with native
> > > > > > persistence. My thinking was that the mechanism itended to
> prevent
> > > data
> > > > > > inconsistencies while nodes with data on the disk will be in the
> > > > process
> > > > > of
> > > > > > joining the cluster.
> > > > > >
> > > > > > Artem, could you please update the docs bringing this to the
> > > attention
> > > > of
> > > > > > the user community?
> > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > >
> > > > > > AG, what if we don't purge data from memory at least for the
> caches
> > > not
> > > > > > backed by the native persistence? Is this a big deal? We can
> > > certainly
> > > > > put
> > > > > > this off by my guts feel we'll return to this question sooner or
> > > later.
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > vololo100@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > For me it looks like some coincidence effect. I understand that
> > we
> > > > get
> > > > > > > such behavior because deactivation works the same way as for
> > > > > > > persistent caches. Was cluster activation/deactivation designed
> > and
> > > > > > > described for in-memory caches? Current behavior sounds for me
> a
> > as
> > > > > > > big risk. I expect a lot of upset users unexpectedly purged all
> > > their
> > > > > > > data.
> > > > > > >
> > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com
> > > > > > > >:
> > > > > > > >
> > > > > > > > Because originally the sole purpose of deactivation was
> > resource
> > > > > > > > deallocation.
> > > > > > > >
> > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dmagda@apache.org
> >:
> > > > > > > >
> > > > > > > > > Such a revelation for me that data is purged from RAM if
> > > someone
> > > > > > > > > deactivates the cluster. Alex, do you remember why we
> decided
> > > to
> > > > > > > implement
> > > > > > > > > it this way initially?
> > > > > > > > >
> > > > > > > > > -
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I agree on CLI and JMX because those interfaces can be
> used
> > > by
> > > > a
> > > > > > > system
> > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > >
> > > > > > > > > > As for the Java API, personally, I find it strange to add
> > > > 'force'
> > > > > > or
> > > > > > > > > > 'confirm' flags to it because it is very unlikely that
> such
> > > an
> > > > > > > invocation
> > > > > > > > > > is done by mistake. Such mistakes are caught during the
> > > testing
> > > > > > > phase and
> > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > anyways.
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > > >
> > >
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Denis Magda <dm...@apache.org>.
Hi Vladimir,
Yes, I'm suggesting us to enable this flag by
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
default instead of introducing --force flag and showing any warnings.
-
Denis
On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com> wrote:
> Hello all.
>
> Denis, which changes exactly? In current implementation of ticket [2] flag
> [1] is checked before requiring --force flag and showing any warnings. Do
> we need to set reuse-memory-on-deactivate to true by default?
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
> [2] https://issues.apache.org/jira/browse/IGNITE-12614
>
>
> вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
>
> > That's the best solution for this scenario. Should we readjust the
> already
> > created ticket [1] suggesting to implement the changes of Alex Scherbakov
> > instead?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> > -
> > Denis
> >
> >
> > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com> wrote:
> >
> > > Folks,
> > >
> > > For a long time we have a flag [1]
> > >
> > > It does almost what we want here.
> > >
> > > I suggest to make this behavior default and rework it to keep data in
> > > memory as well (we already have special "recovery" mode for caches).
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >
> > >
> > >
> > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > >
> > > > I do not mind making this change if we explicitly and clearly define
> > what
> > > > 'new inactive state' means. What should happen if a partition is lost
> > in
> > > > inactive state? What if such node joins back the cluster after? Etc.
> > > >
> > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > >
> > > > > Back up Ivan's opinion here. Initially, the activation/deactivation
> > was
> > > > > created for the baseline topology designed for cases with native
> > > > > persistence. My thinking was that the mechanism itended to prevent
> > data
> > > > > inconsistencies while nodes with data on the disk will be in the
> > > process
> > > > of
> > > > > joining the cluster.
> > > > >
> > > > > Artem, could you please update the docs bringing this to the
> > attention
> > > of
> > > > > the user community?
> > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > >
> > > > > AG, what if we don't purge data from memory at least for the caches
> > not
> > > > > backed by the native persistence? Is this a big deal? We can
> > certainly
> > > > put
> > > > > this off by my guts feel we'll return to this question sooner or
> > later.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> vololo100@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > For me it looks like some coincidence effect. I understand that
> we
> > > get
> > > > > > such behavior because deactivation works the same way as for
> > > > > > persistent caches. Was cluster activation/deactivation designed
> and
> > > > > > described for in-memory caches? Current behavior sounds for me a
> as
> > > > > > big risk. I expect a lot of upset users unexpectedly purged all
> > their
> > > > > > data.
> > > > > >
> > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > > >:
> > > > > > >
> > > > > > > Because originally the sole purpose of deactivation was
> resource
> > > > > > > deallocation.
> > > > > > >
> > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > > > >
> > > > > > > > Such a revelation for me that data is purged from RAM if
> > someone
> > > > > > > > deactivates the cluster. Alex, do you remember why we decided
> > to
> > > > > > implement
> > > > > > > > it this way initially?
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I agree on CLI and JMX because those interfaces can be used
> > by
> > > a
> > > > > > system
> > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > >
> > > > > > > > > As for the Java API, personally, I find it strange to add
> > > 'force'
> > > > > or
> > > > > > > > > 'confirm' flags to it because it is very unlikely that such
> > an
> > > > > > invocation
> > > > > > > > > is done by mistake. Such mistakes are caught during the
> > testing
> > > > > > phase and
> > > > > > > > > developers will end up hard-coding 'true' as a flag
> anyways.
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Vladimir Steshin <vl...@gmail.com>.
Hello all.
Denis, which changes exactly? In current implementation of ticket [2] flag
[1] is checked before requiring --force flag and showing any warnings. Do
we need to set reuse-memory-on-deactivate to true by default?
[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[2] https://issues.apache.org/jira/browse/IGNITE-12614
вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> That's the best solution for this scenario. Should we readjust the already
> created ticket [1] suggesting to implement the changes of Alex Scherbakov
> instead?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>
> -
> Denis
>
>
> On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> wrote:
>
> > Folks,
> >
> > For a long time we have a flag [1]
> >
> > It does almost what we want here.
> >
> > I suggest to make this behavior default and rework it to keep data in
> > memory as well (we already have special "recovery" mode for caches).
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >
> >
> >
> > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > I do not mind making this change if we explicitly and clearly define
> what
> > > 'new inactive state' means. What should happen if a partition is lost
> in
> > > inactive state? What if such node joins back the cluster after? Etc.
> > >
> > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > >
> > > > Back up Ivan's opinion here. Initially, the activation/deactivation
> was
> > > > created for the baseline topology designed for cases with native
> > > > persistence. My thinking was that the mechanism itended to prevent
> data
> > > > inconsistencies while nodes with data on the disk will be in the
> > process
> > > of
> > > > joining the cluster.
> > > >
> > > > Artem, could you please update the docs bringing this to the
> attention
> > of
> > > > the user community?
> > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > >
> > > > AG, what if we don't purge data from memory at least for the caches
> not
> > > > backed by the native persistence? Is this a big deal? We can
> certainly
> > > put
> > > > this off by my guts feel we'll return to this question sooner or
> later.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > > > wrote:
> > > >
> > > > > For me it looks like some coincidence effect. I understand that we
> > get
> > > > > such behavior because deactivation works the same way as for
> > > > > persistent caches. Was cluster activation/deactivation designed and
> > > > > described for in-memory caches? Current behavior sounds for me a as
> > > > > big risk. I expect a lot of upset users unexpectedly purged all
> their
> > > > > data.
> > > > >
> > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > > >:
> > > > > >
> > > > > > Because originally the sole purpose of deactivation was resource
> > > > > > deallocation.
> > > > > >
> > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > > > Such a revelation for me that data is purged from RAM if
> someone
> > > > > > > deactivates the cluster. Alex, do you remember why we decided
> to
> > > > > implement
> > > > > > > it this way initially?
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I agree on CLI and JMX because those interfaces can be used
> by
> > a
> > > > > system
> > > > > > > > administrator and can be invoked by mistake.
> > > > > > > >
> > > > > > > > As for the Java API, personally, I find it strange to add
> > 'force'
> > > > or
> > > > > > > > 'confirm' flags to it because it is very unlikely that such
> an
> > > > > invocation
> > > > > > > > is done by mistake. Such mistakes are caught during the
> testing
> > > > > phase and
> > > > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Denis Magda <dm...@apache.org>.
That's the best solution for this scenario. Should we readjust the already
created ticket [1] suggesting to implement the changes of Alex Scherbakov
instead?
[1] https://issues.apache.org/jira/browse/IGNITE-12614
-
Denis
On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
alexey.scherbakoff@gmail.com> wrote:
> Folks,
>
> For a long time we have a flag [1]
>
> It does almost what we want here.
>
> I suggest to make this behavior default and rework it to keep data in
> memory as well (we already have special "recovery" mode for caches).
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
>
>
> пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > I do not mind making this change if we explicitly and clearly define what
> > 'new inactive state' means. What should happen if a partition is lost in
> > inactive state? What if such node joins back the cluster after? Etc.
> >
> > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> >
> > > Back up Ivan's opinion here. Initially, the activation/deactivation was
> > > created for the baseline topology designed for cases with native
> > > persistence. My thinking was that the mechanism itended to prevent data
> > > inconsistencies while nodes with data on the disk will be in the
> process
> > of
> > > joining the cluster.
> > >
> > > Artem, could you please update the docs bringing this to the attention
> of
> > > the user community?
> > > https://issues.apache.org/jira/browse/IGNITE-12615
> > >
> > > AG, what if we don't purge data from memory at least for the caches not
> > > backed by the native persistence? Is this a big deal? We can certainly
> > put
> > > this off by my guts feel we'll return to this question sooner or later.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > > wrote:
> > >
> > > > For me it looks like some coincidence effect. I understand that we
> get
> > > > such behavior because deactivation works the same way as for
> > > > persistent caches. Was cluster activation/deactivation designed and
> > > > described for in-memory caches? Current behavior sounds for me a as
> > > > big risk. I expect a lot of upset users unexpectedly purged all their
> > > > data.
> > > >
> > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >:
> > > > >
> > > > > Because originally the sole purpose of deactivation was resource
> > > > > deallocation.
> > > > >
> > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > >
> > > > > > Such a revelation for me that data is purged from RAM if someone
> > > > > > deactivates the cluster. Alex, do you remember why we decided to
> > > > implement
> > > > > > it this way initially?
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I agree on CLI and JMX because those interfaces can be used by
> a
> > > > system
> > > > > > > administrator and can be invoked by mistake.
> > > > > > >
> > > > > > > As for the Java API, personally, I find it strange to add
> 'force'
> > > or
> > > > > > > 'confirm' flags to it because it is very unlikely that such an
> > > > invocation
> > > > > > > is done by mistake. Such mistakes are caught during the testing
> > > > phase and
> > > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Alexei Scherbakov <al...@gmail.com>.
Alexey Goncharuk,
I see no issues with "lost" partitions. The same is possible right now in
persistence mode. Node can be killed and data erased while grid is
deactivated.
After re-join it should rebalance itself from existing owners (if any
exists).
вт, 4 февр. 2020 г. в 10:54, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:
> Folks,
>
> For a long time we have a flag [1]
>
> It does almost what we want here.
>
> I suggest to make this behavior default and rework it to keep data in
> memory as well (we already have special "recovery" mode for caches).
>
>
> [1] org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
>
>
> пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
>> I do not mind making this change if we explicitly and clearly define what
>> 'new inactive state' means. What should happen if a partition is lost in
>> inactive state? What if such node joins back the cluster after? Etc.
>>
>> пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>>
>> > Back up Ivan's opinion here. Initially, the activation/deactivation was
>> > created for the baseline topology designed for cases with native
>> > persistence. My thinking was that the mechanism itended to prevent data
>> > inconsistencies while nodes with data on the disk will be in the
>> process of
>> > joining the cluster.
>> >
>> > Artem, could you please update the docs bringing this to the attention
>> of
>> > the user community?
>> > https://issues.apache.org/jira/browse/IGNITE-12615
>> >
>> > AG, what if we don't purge data from memory at least for the caches not
>> > backed by the native persistence? Is this a big deal? We can certainly
>> put
>> > this off by my guts feel we'll return to this question sooner or later.
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
>> > wrote:
>> >
>> > > For me it looks like some coincidence effect. I understand that we get
>> > > such behavior because deactivation works the same way as for
>> > > persistent caches. Was cluster activation/deactivation designed and
>> > > described for in-memory caches? Current behavior sounds for me a as
>> > > big risk. I expect a lot of upset users unexpectedly purged all their
>> > > data.
>> > >
>> > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
>> > alexey.goncharuk@gmail.com
>> > > >:
>> > > >
>> > > > Because originally the sole purpose of deactivation was resource
>> > > > deallocation.
>> > > >
>> > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
>> > > >
>> > > > > Such a revelation for me that data is purged from RAM if someone
>> > > > > deactivates the cluster. Alex, do you remember why we decided to
>> > > implement
>> > > > > it this way initially?
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
>> > > > > alexey.goncharuk@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I agree on CLI and JMX because those interfaces can be used by a
>> > > system
>> > > > > > administrator and can be invoked by mistake.
>> > > > > >
>> > > > > > As for the Java API, personally, I find it strange to add
>> 'force'
>> > or
>> > > > > > 'confirm' flags to it because it is very unlikely that such an
>> > > invocation
>> > > > > > is done by mistake. Such mistakes are caught during the testing
>> > > phase and
>> > > > > > developers will end up hard-coding 'true' as a flag anyways.
>> > > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Ivan Pavlukhin
>> > >
>> >
>>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>
--
Best regards,
Alexei Scherbakov
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Posted by Alexei Scherbakov <al...@gmail.com>.
Folks,
For a long time we have a flag [1]
It does almost what we want here.
I suggest to make this behavior default and rework it to keep data in
memory as well (we already have special "recovery" mode for caches).
[1] org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <al...@gmail.com>:
> I do not mind making this change if we explicitly and clearly define what
> 'new inactive state' means. What should happen if a partition is lost in
> inactive state? What if such node joins back the cluster after? Etc.
>
> пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>
> > Back up Ivan's opinion here. Initially, the activation/deactivation was
> > created for the baseline topology designed for cases with native
> > persistence. My thinking was that the mechanism itended to prevent data
> > inconsistencies while nodes with data on the disk will be in the process
> of
> > joining the cluster.
> >
> > Artem, could you please update the docs bringing this to the attention of
> > the user community?
> > https://issues.apache.org/jira/browse/IGNITE-12615
> >
> > AG, what if we don't purge data from memory at least for the caches not
> > backed by the native persistence? Is this a big deal? We can certainly
> put
> > this off by my guts feel we'll return to this question sooner or later.
> >
> > -
> > Denis
> >
> >
> > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > wrote:
> >
> > > For me it looks like some coincidence effect. I understand that we get
> > > such behavior because deactivation works the same way as for
> > > persistent caches. Was cluster activation/deactivation designed and
> > > described for in-memory caches? Current behavior sounds for me a as
> > > big risk. I expect a lot of upset users unexpectedly purged all their
> > > data.
> > >
> > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > > >
> > > > Because originally the sole purpose of deactivation was resource
> > > > deallocation.
> > > >
> > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > >
> > > > > Such a revelation for me that data is purged from RAM if someone
> > > > > deactivates the cluster. Alex, do you remember why we decided to
> > > implement
> > > > > it this way initially?
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I agree on CLI and JMX because those interfaces can be used by a
> > > system
> > > > > > administrator and can be invoked by mistake.
> > > > > >
> > > > > > As for the Java API, personally, I find it strange to add 'force'
> > or
> > > > > > 'confirm' flags to it because it is very unlikely that such an
> > > invocation
> > > > > > is done by mistake. Such mistakes are caught during the testing
> > > phase and
> > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >
>
--
Best regards,
Alexei Scherbakov