Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Igniters.

I found really confusing results of the simple test.
Data from the in-memory cache are vanished after change cluster state to INACTIVE/ACTIVE.

Is this a bug or expected behavior?


```
public class ClusterDeactivateTest extends GridCommonAbstractTest {
    String name = "my-cache";

    /** */
    @Test
    public void testDataPresent() throws Exception {
        IgniteEx i = startGrid(0);

        i.createCache(name).put(1L, 1L);

        assertEquals(1L, i.cache(name).get(1L));

        i.cluster().state(ClusterState.INACTIVE);
        i.cluster().state(ClusterState.ACTIVE);

        assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
    }
}
```

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Alexey.

The issue with the existing method.

Single implementation for both Ignite#active(boolean) and IgniteMXBean#active(boolean)
Should it continue to silently clears in-memory caches?

> 14 февр. 2020 г., в 15:20, Alexey Goncharuk <al...@gmail.com> написал(а):
> 
> Igniters,
> 
> Do we really need the confirmation flag on the public API? I absolutely
> agree on the CLI and MXBean, but what is the reason for the flag in the
> API? It will be specified at the compile time anyway and does not prevent
> any user error.
> From the implementation point of view I see no contradiction - we can add
> the new method to the MXBean, but nothing forces us to add it to Ignite
> interface - those interfaces are not related.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikita Amelchev <ns...@gmail.com>]

Hi, Igniters.

Note that DeactivateCommand#confirmationPrompt will be ignored in case
of auto confirmation.

I agree that the force flag should be removed when the user data and
datastructures will not be clearing on deactivation. For now, cmd, jmx
and rest interfaces should be covered.

вт, 24 мар. 2020 г. в 15:56, Вячеслав Коптилин <sl...@gmail.com>:
>
> Hello Nikolay,
>
> I am talking about the interactive mode of the control utility, which
> requires explicit confirmation from the user.
> Please take a look at DeactivateCommand#prepareConfirmation and its usages.
> It seems to me, this mode has the same aim as the forceDeactivation flag.
> We can change the message returned by DeactivateCommand#confirmationPrompt
> as follows:
>     "Warning: the command will deactivate the cluster nnn and clear
> in-memory caches (without persistence) including system caches."
>
> What do you think?
>
> Thanks,
> S.
>
> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>
> > Hello, Slava.
> >
> > Are you talking about this commit [1] (sorry for commit message it’s due
> > to the Github issue)?
> >
> > The message for this command for now
> >
> > «Deactivation stopped. Deactivation clears in-memory caches (without
> > persistence) including the system caches.»
> >
> > Is it clear enough?
> >
> > [1]
> > https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >
> >
> > > 24 марта 2020 г., в 13:02, Вячеслав Коптилин <sl...@gmail.com>
> > написал(а):
> > >
> > > Hi Nikolay,
> > >
> > >> 1. We should add —force flag to the command.sh deactivation command.
> > > I just checked and it seems that the deactivation command
> > > (control-utility.sh) already has a confirmation option.
> > > Perhaps, we need to clearly state the consequences of using this command
> > > with in-memory caches.
> > >
> > > Thanks,
> > > S.
> > >
> > > вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
> > >
> > >> Hello, Alexey.
> > >>
> > >> I just repeat our agreement to be on the same page
> > >>
> > >>> The confirmation should only present in the user-facing interfaces.
> > >>
> > >> 1. We should add —force flag to the command.sh deactivation command.
> > >> 2. We should throw the exception if cluster has in-memory caches and
> > >> —force=false.
> > >> 3. We shouldn’t change Java API for deactivation.
> > >>
> > >> Is it correct?
> > >>
> > >>> The DROP TABLE command does not have a "yes I am sure" clause in it
> > >>
> > >> I think it because the command itself has a «DROP» word in it’s name.
> > >> Which clearly explains what will happen on it’s execution.
> > >>
> > >> On the opposite «deactivation» command doesn’t have any sign of
> > >> destructive operation.
> > >> That’s why we should warn user about it’s consequences.
> > >>
> > >>
> > >>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com>
> > >> написал(а):
> > >>>
> > >>> Igniters, Ivan, Nikolay,
> > >>>
> > >>> I am strongly against adding confirmation flags to any kind of APIs,
> > >>> whether we change the deactivation behavior or not (even though I agree
> > >>> that it makes sense to fix the deactivation to not clean up the
> > in-memory
> > >>> data). The confirmation should only present in the user-facing
> > >> interfaces.
> > >>> I cannot recall any software interface which has such a flag. None of
> > the
> > >>> syscalls which delete files (a very destructive operation) have this
> > >> flag.
> > >>> The DROP TABLE command does not have a "yes I am sure" clause in it.
> > As I
> > >>> already mentioned, when used programmatically, most users will likely
> > >>> simply pass 'true' as the new flag because they already know the
> > >> behavior.
> > >>> This is a clear sign of a bad design choice.
> > >>>
> > >>> On top of that, given that it is our intention to change the behavior
> > of
> > >>> deactivation to not loose the in-memory data, it does not make sense to
> > >> me
> > >>> to change the API.
> > >>
> > >>
> >
> >



-- 
Best wishes,
Amelchev Nikita

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hello Vladimir,

> onto anolugous
>  @param forceDeactivation If {@code true}, cluster deactivation will be
forced. //Deactivation clears in-memory caches (without persistence)
including the system caches.
My idea was about providing a link to one place with a complete description
instead of copy&paste.

> We might include this fix in the last [1].
It is up to you, Vladimir.

Thanks,
S.



пт, 3 апр. 2020 г. в 15:42, Vladimir Steshin <vl...@gmail.com>:

> Slava, hello.
>
>
> All right, since we have in public API several
>
> /* Deactivation clears in-memory caches (without persistence) including
> the system caches./
>
> We should change in the internals
>
> /    @param forceDeactivation If {@code true}, cluster deactivation will
> be forced./
>
> onto anolugous
>
> /    @param forceDeactivation If {@code true}, cluster deactivation will
> be forced. //Deactivation clears in-memory caches (without persistence)
> including the system caches./
>
> //
>
> We might include this fix in the last [1]. WDYT, can we proceed with [1]
> then ?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12779
>
>
>
> 02.04.2020 19:58, Вячеслав Коптилин пишет:
> > Hi Vladimir,
> >
> >> There are about 15 places in inner logic with this description.
> >> I propose balance between code base size and comment completeness.
> > I agree with Iva and I also think that this approach is not so good.
> > Perhaps we can add just a link to the one method which will provide a
> > comprehensive description, something like as follows
> > @param forceDeactivation {@code true} if cluster deactivation should be
> > forced. Please take a look at {@link IgniteCluster#state(ClusterState
> > newState, boolean force)} for the details.
> >
> > What do you think?
> >
> > Thanks,
> > Slava.
> >
> > чт, 2 апр. 2020 г. в 18:47, Vladimir Steshin <vl...@gmail.com>:
> >
> >> Ivan, hello.
> >>
> >> Thanks. I vote for keeping the comments as they are now :)
> >>
> >> Igniters, it seems we are agreed to merge [1]. And the ticked s to be
> >> reverted in future with new designed solution of keeping in-memory data
> >> after deactivation.
> >>
> >> Right?
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-12779
> >>
> >>
> >> 01.04.2020 20:20, Ivan Rakov пишет:
> >>> I don't think that making javadocs more descriptive can be considered
> as
> >>> harmful code base enlargement.
> >>> I'd recommend to extend the docs, but the last word is yours ;)
> >>>
> >>> On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin <vl...@gmail.com>
> >> wrote:
> >>>> Ivan, hi.
> >>>>
> >>>> I absolutely agree this particular description is not enough to see
> the
> >>>> deactivation issue. I also vote for brief code.
> >>>>
> >>>> There are about 15 places in inner logic with this description. I
> >>>> propose balance between code base size and comment completeness.
> >>>>
> >>>> Should we enlarge code even if we got several full descriptions?
> >>>>
> >>>>
> >>>> 30.03.2020 20:02, Ivan Rakov пишет:
> >>>>> Vladimir,
> >>>>>
> >>>>> @param forceDeactivation If {@code true}, cluster deactivation will
> be
> >>>>>> forced.
> >>>>> It's true that it's possible to infer semantics of forced
> deactivation
> >>>> from
> >>>>> other parts of API. I just wanted to highlight that exactly this
> >>>>> description explains something that can be guessed by the parameter
> >> name.
> >>>>> I suppose to shorten the lookup path and shed a light on deactivation
> >>>>> semantics a bit:
> >>>>>
> >>>>>> @param forceDeactivation If {@code true}, cluster will be
> deactivated
> >>>> even
> >>>>>> if running in-memory caches are present. All data in the
> corresponding
> >>>>>> caches will be vanished as a result.
> >>>>> Does this make sense?
> >>>>>
> >>>>> On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <
> vladsz83@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Ivan, hi.
> >>>>>>
> >>>>>>
> >>>>>> 1) >>> Is it correct? If we are on the same page, let's proceed this
> >> way
> >>>>>> It is correct.
> >>>>>>
> >>>>>>
> >>>>>> 2) - In many places in the code I can see the following javadoc
> >>>>>>
> >>>>>>>      @param forceDeactivation If {@code true}, cluster deactivation
> >> will
> >>>> be
> >>>>>> forced.
> >>>>>>
> >>>>>> In the internal params/flags. You can also find /@see
> >>>>>> ClusterState#INACTIVE/ and full description with several public
> APIs (
> >>>>>> like /Ignite.active(boolean)/ ):
> >>>>>>
> >>>>>> //
> >>>>>>
> >>>>>> /* <p>/
> >>>>>>
> >>>>>> //
> >>>>>>
> >>>>>> /* <b>NOTE:</b>/
> >>>>>>
> >>>>>> //
> >>>>>>
> >>>>>> /* Deactivation clears in-memory caches (without persistence)
> >> including
> >>>>>> the system caches./
> >>>>>>
> >>>>>> Should be enough. Is not?
> >>>>>>
> >>>>>>
> >>>>>> 27.03.2020 10:51, Ivan Rakov пишет:
> >>>>>>> Vladimir, Igniters,
> >>>>>>>
> >>>>>>> Let's emphasize our final plan.
> >>>>>>>
> >>>>>>> We are going to add --force flags that will be necessary to pass
> for
> >> a
> >>>>>>> deactivation if there are in-memory caches to:
> >>>>>>> 1) Rest API (already implemented in [1])
> >>>>>>> 2) Command line utility (already implemented in [1])
> >>>>>>> 3) JMX bean (going to be implemented in [2])
> >>>>>>> We are *not* going to change IgniteCluster or any other thick Java
> >> API,
> >>>>>>> thus we are *not* going to merge [3].
> >>>>>>> We plan to *fully rollback* [1] and [2] once cache data survival
> >> after
> >>>>>>> activation-deactivation cycle will be implemented.
> >>>>>>>
> >>>>>>> Is it correct? If we are on the same page, let's proceed this way.
> >>>>>>> I propose to:
> >>>>>>> - Create a JIRA issue for in-memory-data-safe deactivation
> (possibly,
> >>>>>>> without IEP and detailed design so far)
> >>>>>>> - Describe in the issue description what exact parts of API should
> be
> >>>>>>> removed under the issue scope.
> >>>>>>>
> >>>>>>> Also, a few questions on already merged [1]:
> >>>>>>> - We have removed GridClientClusterState#state(ClusterState) from
> >> Java
> >>>>>>> client API. Is it a legitimate thing to do? Don't we have to
> support
> >>>> API
> >>>>>>> compatibility for thin clients as well?
> >>>>>>> - In many places in the code I can see the following javadoc
> >>>>>>>
> >>>>>>>>      @param forceDeactivation If {@code true}, cluster
> deactivation
> >> will
> >>>>>> be forced.
> >>>>>>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
> >>>>>> describe
> >>>>>>> in which cases deactivation won't happen unless it's forced and
> which
> >>>>>>> impact forced deactivation will bring on the system.
> >>>>>>>
> >>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> >>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> >>>>>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>>>>
> >>>>>>> --
> >>>>>>> Ivan
> >>>>>>>
> >>>>>>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <
> vladsz83@gmail.com
> >>>>>> wrote:
> >>>>>>>> Hi, Igniters.
> >>>>>>>>
> >>>>>>>> I'd like to remind you that cluster can be deactivated by user
> with
> >> 3
> >>>>>>>> utilities: control.sh, *JMX and the REST*. Proposed in [1]
> solution
> >> is
> >>>>>>>> not about control.sh. It suggests same approach regardless of the
> >>>>>>>> utility user executes. The task touches *only* *API of the user
> >>>> calls*,
> >>>>>>>> not the internal APIs.
> >>>>>>>>
> >>>>>>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken
> >> into
> >>>>>>>> account for control.sh are:
> >>>>>>>>
> >>>>>>>> -Various commands widely use “--yes” just to start. Even not
> >> dangerous
> >>>>>>>> ones require “--yes” to begin. “--force” is dedicated for
> *harmless
> >>>>>>>> actions*.
> >>>>>>>>
> >>>>>>>> -Checking of probable data erasure works after command start and
> >>>>>>>> “--force” may not be required at all.
> >>>>>>>>
> >>>>>>>> -There are also JMX and REST. They have no “—yes” but should work
> >>>> alike.
> >>>>>>>>          To get the deactivation safe I propose to merge last
> ticket
> >>>> with
> >>>>>>>> the JMX fixes [2]. In future releases, I believe, we should
> estimate
> >>>>>>>> jobs and fix memory erasure in general. For now, let’s prevent it.
> >>>> WDYT?
> >>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>>>>>
> >>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
> >>>>>>>>> Hello Nikolay,
> >>>>>>>>>
> >>>>>>>>> I am talking about the interactive mode of the control utility,
> >> which
> >>>>>>>>> requires explicit confirmation from the user.
> >>>>>>>>> Please take a look at DeactivateCommand#prepareConfirmation and
> its
> >>>>>>>> usages.
> >>>>>>>>> It seems to me, this mode has the same aim as the
> forceDeactivation
> >>>>>> flag.
> >>>>>>>>> We can change the message returned by
> >>>>>>>> DeactivateCommand#confirmationPrompt
> >>>>>>>>> as follows:
> >>>>>>>>>          "Warning: the command will deactivate the cluster nnn
> and
> >>>> clear
> >>>>>>>>> in-memory caches (without persistence) including system caches."
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> S.
> >>>>>>>>>
> >>>>>>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <
> nizhikov@apache.org
> >>> :
> >>>>>>>>>> Hello, Slava.
> >>>>>>>>>>
> >>>>>>>>>> Are you talking about this commit [1] (sorry for commit message
> >> it’s
> >>>>>> due
> >>>>>>>>>> to the Github issue)?
> >>>>>>>>>>
> >>>>>>>>>> The message for this command for now
> >>>>>>>>>>
> >>>>>>>>>> «Deactivation stopped. Deactivation clears in-memory caches
> >> (without
> >>>>>>>>>> persistence) including the system caches.»
> >>>>>>>>>>
> >>>>>>>>>> Is it clear enough?
> >>>>>>>>>>
> >>>>>>>>>> [1]
> >>>>>>>>>>
> >>
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >>>>>>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
> >>>>>> slava.koptilin@gmail.com
> >>>>>>>>>> написал(а):
> >>>>>>>>>>> Hi Nikolay,
> >>>>>>>>>>>
> >>>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
> >>>> command.
> >>>>>>>>>>> I just checked and it seems that the deactivation command
> >>>>>>>>>>> (control-utility.sh) already has a confirmation option.
> >>>>>>>>>>> Perhaps, we need to clearly state the consequences of using
> this
> >>>>>>>> command
> >>>>>>>>>>> with in-memory caches.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> S.
> >>>>>>>>>>>
> >>>>>>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <
> >> nizhikov@apache.org
> >>>>> :
> >>>>>>>>>>>> Hello, Alexey.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I just repeat our agreement to be on the same page
> >>>>>>>>>>>>
> >>>>>>>>>>>>> The confirmation should only present in the user-facing
> >>>> interfaces.
> >>>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
> >>>> command.
> >>>>>>>>>>>> 2. We should throw the exception if cluster has in-memory
> caches
> >>>> and
> >>>>>>>>>>>> —force=false.
> >>>>>>>>>>>> 3. We shouldn’t change Java API for deactivation.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is it correct?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
> >> in
> >>>> it
> >>>>>>>>>>>> I think it because the command itself has a «DROP» word in
> it’s
> >>>>>> name.
> >>>>>>>>>>>> Which clearly explains what will happen on it’s execution.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On the opposite «deactivation» command doesn’t have any sign
> of
> >>>>>>>>>>>> destructive operation.
> >>>>>>>>>>>> That’s why we should warn user about it’s consequences.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> >>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>> написал(а):
> >>>>>>>>>>>>> Igniters, Ivan, Nikolay,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I am strongly against adding confirmation flags to any kind
> of
> >>>>>> APIs,
> >>>>>>>>>>>>> whether we change the deactivation behavior or not (even
> >> though I
> >>>>>>>> agree
> >>>>>>>>>>>>> that it makes sense to fix the deactivation to not clean up
> the
> >>>>>>>>>> in-memory
> >>>>>>>>>>>>> data). The confirmation should only present in the
> user-facing
> >>>>>>>>>>>> interfaces.
> >>>>>>>>>>>>> I cannot recall any software interface which has such a flag.
> >>>> None
> >>>>>> of
> >>>>>>>>>> the
> >>>>>>>>>>>>> syscalls which delete files (a very destructive operation)
> have
> >>>>>> this
> >>>>>>>>>>>> flag.
> >>>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
> >> in
> >>>>>> it.
> >>>>>>>>>> As I
> >>>>>>>>>>>>> already mentioned, when used programmatically, most users
> will
> >>>>>> likely
> >>>>>>>>>>>>> simply pass 'true' as the new flag because they already know
> >> the
> >>>>>>>>>>>> behavior.
> >>>>>>>>>>>>> This is a clear sign of a bad design choice.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On top of that, given that it is our intention to change the
> >>>>>> behavior
> >>>>>>>>>> of
> >>>>>>>>>>>>> deactivation to not loose the in-memory data, it does not
> make
> >>>>>> sense
> >>>>>>>> to
> >>>>>>>>>>>> me
> >>>>>>>>>>>>> to change the API.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Slava, hello.


All right, since we have in public API several

/* Deactivation clears in-memory caches (without persistence) including 
the system caches./

We should change in the internals

/    @param forceDeactivation If {@code true}, cluster deactivation will 
be forced./

onto anolugous

/    @param forceDeactivation If {@code true}, cluster deactivation will 
be forced. //Deactivation clears in-memory caches (without persistence) 
including the system caches./

//

We might include this fix in the last [1]. WDYT, can we proceed with [1] 
then ?

[1] https://issues.apache.org/jira/browse/IGNITE-12779



02.04.2020 19:58, Вячеслав Коптилин пишет:
> Hi Vladimir,
>
>> There are about 15 places in inner logic with this description.
>> I propose balance between code base size and comment completeness.
> I agree with Iva and I also think that this approach is not so good.
> Perhaps we can add just a link to the one method which will provide a
> comprehensive description, something like as follows
> @param forceDeactivation {@code true} if cluster deactivation should be
> forced. Please take a look at {@link IgniteCluster#state(ClusterState
> newState, boolean force)} for the details.
>
> What do you think?
>
> Thanks,
> Slava.
>
> чт, 2 апр. 2020 г. в 18:47, Vladimir Steshin <vl...@gmail.com>:
>
>> Ivan, hello.
>>
>> Thanks. I vote for keeping the comments as they are now :)
>>
>> Igniters, it seems we are agreed to merge [1]. And the ticked s to be
>> reverted in future with new designed solution of keeping in-memory data
>> after deactivation.
>>
>> Right?
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-12779
>>
>>
>> 01.04.2020 20:20, Ivan Rakov пишет:
>>> I don't think that making javadocs more descriptive can be considered as
>>> harmful code base enlargement.
>>> I'd recommend to extend the docs, but the last word is yours ;)
>>>
>>> On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin <vl...@gmail.com>
>> wrote:
>>>> Ivan, hi.
>>>>
>>>> I absolutely agree this particular description is not enough to see the
>>>> deactivation issue. I also vote for brief code.
>>>>
>>>> There are about 15 places in inner logic with this description. I
>>>> propose balance between code base size and comment completeness.
>>>>
>>>> Should we enlarge code even if we got several full descriptions?
>>>>
>>>>
>>>> 30.03.2020 20:02, Ivan Rakov пишет:
>>>>> Vladimir,
>>>>>
>>>>> @param forceDeactivation If {@code true}, cluster deactivation will be
>>>>>> forced.
>>>>> It's true that it's possible to infer semantics of forced deactivation
>>>> from
>>>>> other parts of API. I just wanted to highlight that exactly this
>>>>> description explains something that can be guessed by the parameter
>> name.
>>>>> I suppose to shorten the lookup path and shed a light on deactivation
>>>>> semantics a bit:
>>>>>
>>>>>> @param forceDeactivation If {@code true}, cluster will be deactivated
>>>> even
>>>>>> if running in-memory caches are present. All data in the corresponding
>>>>>> caches will be vanished as a result.
>>>>> Does this make sense?
>>>>>
>>>>> On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Ivan, hi.
>>>>>>
>>>>>>
>>>>>> 1) >>> Is it correct? If we are on the same page, let's proceed this
>> way
>>>>>> It is correct.
>>>>>>
>>>>>>
>>>>>> 2) - In many places in the code I can see the following javadoc
>>>>>>
>>>>>>>      @param forceDeactivation If {@code true}, cluster deactivation
>> will
>>>> be
>>>>>> forced.
>>>>>>
>>>>>> In the internal params/flags. You can also find /@see
>>>>>> ClusterState#INACTIVE/ and full description with several public APIs (
>>>>>> like /Ignite.active(boolean)/ ):
>>>>>>
>>>>>> //
>>>>>>
>>>>>> /* <p>/
>>>>>>
>>>>>> //
>>>>>>
>>>>>> /* <b>NOTE:</b>/
>>>>>>
>>>>>> //
>>>>>>
>>>>>> /* Deactivation clears in-memory caches (without persistence)
>> including
>>>>>> the system caches./
>>>>>>
>>>>>> Should be enough. Is not?
>>>>>>
>>>>>>
>>>>>> 27.03.2020 10:51, Ivan Rakov пишет:
>>>>>>> Vladimir, Igniters,
>>>>>>>
>>>>>>> Let's emphasize our final plan.
>>>>>>>
>>>>>>> We are going to add --force flags that will be necessary to pass for
>> a
>>>>>>> deactivation if there are in-memory caches to:
>>>>>>> 1) Rest API (already implemented in [1])
>>>>>>> 2) Command line utility (already implemented in [1])
>>>>>>> 3) JMX bean (going to be implemented in [2])
>>>>>>> We are *not* going to change IgniteCluster or any other thick Java
>> API,
>>>>>>> thus we are *not* going to merge [3].
>>>>>>> We plan to *fully rollback* [1] and [2] once cache data survival
>> after
>>>>>>> activation-deactivation cycle will be implemented.
>>>>>>>
>>>>>>> Is it correct? If we are on the same page, let's proceed this way.
>>>>>>> I propose to:
>>>>>>> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
>>>>>>> without IEP and detailed design so far)
>>>>>>> - Describe in the issue description what exact parts of API should be
>>>>>>> removed under the issue scope.
>>>>>>>
>>>>>>> Also, a few questions on already merged [1]:
>>>>>>> - We have removed GridClientClusterState#state(ClusterState) from
>> Java
>>>>>>> client API. Is it a legitimate thing to do? Don't we have to support
>>>> API
>>>>>>> compatibility for thin clients as well?
>>>>>>> - In many places in the code I can see the following javadoc
>>>>>>>
>>>>>>>>      @param forceDeactivation If {@code true}, cluster deactivation
>> will
>>>>>> be forced.
>>>>>>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
>>>>>> describe
>>>>>>> in which cases deactivation won't happen unless it's forced and which
>>>>>>> impact forced deactivation will bring on the system.
>>>>>>>
>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
>>>>>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
>>>>>>>
>>>>>>> --
>>>>>>> Ivan
>>>>>>>
>>>>>>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vladsz83@gmail.com
>>>>>> wrote:
>>>>>>>> Hi, Igniters.
>>>>>>>>
>>>>>>>> I'd like to remind you that cluster can be deactivated by user with
>> 3
>>>>>>>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution
>> is
>>>>>>>> not about control.sh. It suggests same approach regardless of the
>>>>>>>> utility user executes. The task touches *only* *API of the user
>>>> calls*,
>>>>>>>> not the internal APIs.
>>>>>>>>
>>>>>>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken
>> into
>>>>>>>> account for control.sh are:
>>>>>>>>
>>>>>>>> -Various commands widely use “--yes” just to start. Even not
>> dangerous
>>>>>>>> ones require “--yes” to begin. “--force” is dedicated for *harmless
>>>>>>>> actions*.
>>>>>>>>
>>>>>>>> -Checking of probable data erasure works after command start and
>>>>>>>> “--force” may not be required at all.
>>>>>>>>
>>>>>>>> -There are also JMX and REST. They have no “—yes” but should work
>>>> alike.
>>>>>>>>          To get the deactivation safe I propose to merge last ticket
>>>> with
>>>>>>>> the JMX fixes [2]. In future releases, I believe, we should estimate
>>>>>>>> jobs and fix memory erasure in general. For now, let’s prevent it.
>>>> WDYT?
>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>>>>>>>>
>>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>>>>>>>>
>>>>>>>>
>>>>>>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
>>>>>>>>> Hello Nikolay,
>>>>>>>>>
>>>>>>>>> I am talking about the interactive mode of the control utility,
>> which
>>>>>>>>> requires explicit confirmation from the user.
>>>>>>>>> Please take a look at DeactivateCommand#prepareConfirmation and its
>>>>>>>> usages.
>>>>>>>>> It seems to me, this mode has the same aim as the forceDeactivation
>>>>>> flag.
>>>>>>>>> We can change the message returned by
>>>>>>>> DeactivateCommand#confirmationPrompt
>>>>>>>>> as follows:
>>>>>>>>>          "Warning: the command will deactivate the cluster nnn and
>>>> clear
>>>>>>>>> in-memory caches (without persistence) including system caches."
>>>>>>>>>
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> S.
>>>>>>>>>
>>>>>>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <nizhikov@apache.org
>>> :
>>>>>>>>>> Hello, Slava.
>>>>>>>>>>
>>>>>>>>>> Are you talking about this commit [1] (sorry for commit message
>> it’s
>>>>>> due
>>>>>>>>>> to the Github issue)?
>>>>>>>>>>
>>>>>>>>>> The message for this command for now
>>>>>>>>>>
>>>>>>>>>> «Deactivation stopped. Deactivation clears in-memory caches
>> (without
>>>>>>>>>> persistence) including the system caches.»
>>>>>>>>>>
>>>>>>>>>> Is it clear enough?
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>>>>>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
>>>>>> slava.koptilin@gmail.com
>>>>>>>>>> написал(а):
>>>>>>>>>>> Hi Nikolay,
>>>>>>>>>>>
>>>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
>>>> command.
>>>>>>>>>>> I just checked and it seems that the deactivation command
>>>>>>>>>>> (control-utility.sh) already has a confirmation option.
>>>>>>>>>>> Perhaps, we need to clearly state the consequences of using this
>>>>>>>> command
>>>>>>>>>>> with in-memory caches.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> S.
>>>>>>>>>>>
>>>>>>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <
>> nizhikov@apache.org
>>>>> :
>>>>>>>>>>>> Hello, Alexey.
>>>>>>>>>>>>
>>>>>>>>>>>> I just repeat our agreement to be on the same page
>>>>>>>>>>>>
>>>>>>>>>>>>> The confirmation should only present in the user-facing
>>>> interfaces.
>>>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
>>>> command.
>>>>>>>>>>>> 2. We should throw the exception if cluster has in-memory caches
>>>> and
>>>>>>>>>>>> —force=false.
>>>>>>>>>>>> 3. We shouldn’t change Java API for deactivation.
>>>>>>>>>>>>
>>>>>>>>>>>> Is it correct?
>>>>>>>>>>>>
>>>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
>> in
>>>> it
>>>>>>>>>>>> I think it because the command itself has a «DROP» word in it’s
>>>>>> name.
>>>>>>>>>>>> Which clearly explains what will happen on it’s execution.
>>>>>>>>>>>>
>>>>>>>>>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>>>>>>>>>> destructive operation.
>>>>>>>>>>>> That’s why we should warn user about it’s consequences.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>> написал(а):
>>>>>>>>>>>>> Igniters, Ivan, Nikolay,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am strongly against adding confirmation flags to any kind of
>>>>>> APIs,
>>>>>>>>>>>>> whether we change the deactivation behavior or not (even
>> though I
>>>>>>>> agree
>>>>>>>>>>>>> that it makes sense to fix the deactivation to not clean up the
>>>>>>>>>> in-memory
>>>>>>>>>>>>> data). The confirmation should only present in the user-facing
>>>>>>>>>>>> interfaces.
>>>>>>>>>>>>> I cannot recall any software interface which has such a flag.
>>>> None
>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>> syscalls which delete files (a very destructive operation) have
>>>>>> this
>>>>>>>>>>>> flag.
>>>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
>> in
>>>>>> it.
>>>>>>>>>> As I
>>>>>>>>>>>>> already mentioned, when used programmatically, most users will
>>>>>> likely
>>>>>>>>>>>>> simply pass 'true' as the new flag because they already know
>> the
>>>>>>>>>>>> behavior.
>>>>>>>>>>>>> This is a clear sign of a bad design choice.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On top of that, given that it is our intention to change the
>>>>>> behavior
>>>>>>>>>> of
>>>>>>>>>>>>> deactivation to not loose the in-memory data, it does not make
>>>>>> sense
>>>>>>>> to
>>>>>>>>>>>> me
>>>>>>>>>>>>> to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hi Vladimir,

> There are about 15 places in inner logic with this description.
> I propose balance between code base size and comment completeness.
I agree with Iva and I also think that this approach is not so good.
Perhaps we can add just a link to the one method which will provide a
comprehensive description, something like as follows
@param forceDeactivation {@code true} if cluster deactivation should be
forced. Please take a look at {@link IgniteCluster#state(ClusterState
newState, boolean force)} for the details.

What do you think?

Thanks,
Slava.

чт, 2 апр. 2020 г. в 18:47, Vladimir Steshin <vl...@gmail.com>:

> Ivan, hello.
>
> Thanks. I vote for keeping the comments as they are now :)
>
> Igniters, it seems we are agreed to merge [1]. And the ticked s to be
> reverted in future with new designed solution of keeping in-memory data
> after deactivation.
>
> Right?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12779
>
>
> 01.04.2020 20:20, Ivan Rakov пишет:
> > I don't think that making javadocs more descriptive can be considered as
> > harmful code base enlargement.
> > I'd recommend to extend the docs, but the last word is yours ;)
> >
> > On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin <vl...@gmail.com>
> wrote:
> >
> >> Ivan, hi.
> >>
> >> I absolutely agree this particular description is not enough to see the
> >> deactivation issue. I also vote for brief code.
> >>
> >> There are about 15 places in inner logic with this description. I
> >> propose balance between code base size and comment completeness.
> >>
> >> Should we enlarge code even if we got several full descriptions?
> >>
> >>
> >> 30.03.2020 20:02, Ivan Rakov пишет:
> >>> Vladimir,
> >>>
> >>> @param forceDeactivation If {@code true}, cluster deactivation will be
> >>>> forced.
> >>> It's true that it's possible to infer semantics of forced deactivation
> >> from
> >>> other parts of API. I just wanted to highlight that exactly this
> >>> description explains something that can be guessed by the parameter
> name.
> >>> I suppose to shorten the lookup path and shed a light on deactivation
> >>> semantics a bit:
> >>>
> >>>> @param forceDeactivation If {@code true}, cluster will be deactivated
> >> even
> >>>> if running in-memory caches are present. All data in the corresponding
> >>>> caches will be vanished as a result.
> >>> Does this make sense?
> >>>
> >>> On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
> >>> wrote:
> >>>
> >>>> Ivan, hi.
> >>>>
> >>>>
> >>>> 1) >>> Is it correct? If we are on the same page, let's proceed this
> way
> >>>>
> >>>> It is correct.
> >>>>
> >>>>
> >>>> 2) - In many places in the code I can see the following javadoc
> >>>>
> >>>>>     @param forceDeactivation If {@code true}, cluster deactivation
> will
> >> be
> >>>> forced.
> >>>>
> >>>> In the internal params/flags. You can also find /@see
> >>>> ClusterState#INACTIVE/ and full description with several public APIs (
> >>>> like /Ignite.active(boolean)/ ):
> >>>>
> >>>> //
> >>>>
> >>>> /* <p>/
> >>>>
> >>>> //
> >>>>
> >>>> /* <b>NOTE:</b>/
> >>>>
> >>>> //
> >>>>
> >>>> /* Deactivation clears in-memory caches (without persistence)
> including
> >>>> the system caches./
> >>>>
> >>>> Should be enough. Is not?
> >>>>
> >>>>
> >>>> 27.03.2020 10:51, Ivan Rakov пишет:
> >>>>> Vladimir, Igniters,
> >>>>>
> >>>>> Let's emphasize our final plan.
> >>>>>
> >>>>> We are going to add --force flags that will be necessary to pass for
> a
> >>>>> deactivation if there are in-memory caches to:
> >>>>> 1) Rest API (already implemented in [1])
> >>>>> 2) Command line utility (already implemented in [1])
> >>>>> 3) JMX bean (going to be implemented in [2])
> >>>>> We are *not* going to change IgniteCluster or any other thick Java
> API,
> >>>>> thus we are *not* going to merge [3].
> >>>>> We plan to *fully rollback* [1] and [2] once cache data survival
> after
> >>>>> activation-deactivation cycle will be implemented.
> >>>>>
> >>>>> Is it correct? If we are on the same page, let's proceed this way.
> >>>>> I propose to:
> >>>>> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
> >>>>> without IEP and detailed design so far)
> >>>>> - Describe in the issue description what exact parts of API should be
> >>>>> removed under the issue scope.
> >>>>>
> >>>>> Also, a few questions on already merged [1]:
> >>>>> - We have removed GridClientClusterState#state(ClusterState) from
> Java
> >>>>> client API. Is it a legitimate thing to do? Don't we have to support
> >> API
> >>>>> compatibility for thin clients as well?
> >>>>> - In many places in the code I can see the following javadoc
> >>>>>
> >>>>>>     @param forceDeactivation If {@code true}, cluster deactivation
> will
> >>>> be forced.
> >>>>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
> >>>> describe
> >>>>> in which cases deactivation won't happen unless it's forced and which
> >>>>> impact forced deactivation will bring on the system.
> >>>>>
> >>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> >>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> >>>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>>
> >>>>> --
> >>>>> Ivan
> >>>>>
> >>>>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vladsz83@gmail.com
> >
> >>>> wrote:
> >>>>>> Hi, Igniters.
> >>>>>>
> >>>>>> I'd like to remind you that cluster can be deactivated by user with
> 3
> >>>>>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution
> is
> >>>>>> not about control.sh. It suggests same approach regardless of the
> >>>>>> utility user executes. The task touches *only* *API of the user
> >> calls*,
> >>>>>> not the internal APIs.
> >>>>>>
> >>>>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken
> into
> >>>>>> account for control.sh are:
> >>>>>>
> >>>>>> -Various commands widely use “--yes” just to start. Even not
> dangerous
> >>>>>> ones require “--yes” to begin. “--force” is dedicated for *harmless
> >>>>>> actions*.
> >>>>>>
> >>>>>> -Checking of probable data erasure works after command start and
> >>>>>> “--force” may not be required at all.
> >>>>>>
> >>>>>> -There are also JMX and REST. They have no “—yes” but should work
> >> alike.
> >>>>>>         To get the deactivation safe I propose to merge last ticket
> >> with
> >>>>>> the JMX fixes [2]. In future releases, I believe, we should estimate
> >>>>>> jobs and fix memory erasure in general. For now, let’s prevent it.
> >> WDYT?
> >>>>>>
> >>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>>>
> >>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
> >>>>>>
> >>>>>>
> >>>>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
> >>>>>>> Hello Nikolay,
> >>>>>>>
> >>>>>>> I am talking about the interactive mode of the control utility,
> which
> >>>>>>> requires explicit confirmation from the user.
> >>>>>>> Please take a look at DeactivateCommand#prepareConfirmation and its
> >>>>>> usages.
> >>>>>>> It seems to me, this mode has the same aim as the forceDeactivation
> >>>> flag.
> >>>>>>> We can change the message returned by
> >>>>>> DeactivateCommand#confirmationPrompt
> >>>>>>> as follows:
> >>>>>>>         "Warning: the command will deactivate the cluster nnn and
> >> clear
> >>>>>>> in-memory caches (without persistence) including system caches."
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> S.
> >>>>>>>
> >>>>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <nizhikov@apache.org
> >:
> >>>>>>>
> >>>>>>>> Hello, Slava.
> >>>>>>>>
> >>>>>>>> Are you talking about this commit [1] (sorry for commit message
> it’s
> >>>> due
> >>>>>>>> to the Github issue)?
> >>>>>>>>
> >>>>>>>> The message for this command for now
> >>>>>>>>
> >>>>>>>> «Deactivation stopped. Deactivation clears in-memory caches
> (without
> >>>>>>>> persistence) including the system caches.»
> >>>>>>>>
> >>>>>>>> Is it clear enough?
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >>>>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
> >>>> slava.koptilin@gmail.com
> >>>>>>>> написал(а):
> >>>>>>>>> Hi Nikolay,
> >>>>>>>>>
> >>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
> >> command.
> >>>>>>>>> I just checked and it seems that the deactivation command
> >>>>>>>>> (control-utility.sh) already has a confirmation option.
> >>>>>>>>> Perhaps, we need to clearly state the consequences of using this
> >>>>>> command
> >>>>>>>>> with in-memory caches.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> S.
> >>>>>>>>>
> >>>>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <
> nizhikov@apache.org
> >>> :
> >>>>>>>>>> Hello, Alexey.
> >>>>>>>>>>
> >>>>>>>>>> I just repeat our agreement to be on the same page
> >>>>>>>>>>
> >>>>>>>>>>> The confirmation should only present in the user-facing
> >> interfaces.
> >>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
> >> command.
> >>>>>>>>>> 2. We should throw the exception if cluster has in-memory caches
> >> and
> >>>>>>>>>> —force=false.
> >>>>>>>>>> 3. We shouldn’t change Java API for deactivation.
> >>>>>>>>>>
> >>>>>>>>>> Is it correct?
> >>>>>>>>>>
> >>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
> in
> >> it
> >>>>>>>>>> I think it because the command itself has a «DROP» word in it’s
> >>>> name.
> >>>>>>>>>> Which clearly explains what will happen on it’s execution.
> >>>>>>>>>>
> >>>>>>>>>> On the opposite «deactivation» command doesn’t have any sign of
> >>>>>>>>>> destructive operation.
> >>>>>>>>>> That’s why we should warn user about it’s consequences.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> >>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>> написал(а):
> >>>>>>>>>>> Igniters, Ivan, Nikolay,
> >>>>>>>>>>>
> >>>>>>>>>>> I am strongly against adding confirmation flags to any kind of
> >>>> APIs,
> >>>>>>>>>>> whether we change the deactivation behavior or not (even
> though I
> >>>>>> agree
> >>>>>>>>>>> that it makes sense to fix the deactivation to not clean up the
> >>>>>>>> in-memory
> >>>>>>>>>>> data). The confirmation should only present in the user-facing
> >>>>>>>>>> interfaces.
> >>>>>>>>>>> I cannot recall any software interface which has such a flag.
> >> None
> >>>> of
> >>>>>>>> the
> >>>>>>>>>>> syscalls which delete files (a very destructive operation) have
> >>>> this
> >>>>>>>>>> flag.
> >>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause
> in
> >>>> it.
> >>>>>>>> As I
> >>>>>>>>>>> already mentioned, when used programmatically, most users will
> >>>> likely
> >>>>>>>>>>> simply pass 'true' as the new flag because they already know
> the
> >>>>>>>>>> behavior.
> >>>>>>>>>>> This is a clear sign of a bad design choice.
> >>>>>>>>>>>
> >>>>>>>>>>> On top of that, given that it is our intention to change the
> >>>> behavior
> >>>>>>>> of
> >>>>>>>>>>> deactivation to not loose the in-memory data, it does not make
> >>>> sense
> >>>>>> to
> >>>>>>>>>> me
> >>>>>>>>>>> to change the API.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Ivan, hello.

Thanks. I vote for keeping the comments as they are now :)

Igniters, it seems we are agreed to merge [1]. And the ticked s to be 
reverted in future with new designed solution of keeping in-memory data 
after deactivation.

Right?


[1] https://issues.apache.org/jira/browse/IGNITE-12779


01.04.2020 20:20, Ivan Rakov пишет:
> I don't think that making javadocs more descriptive can be considered as
> harmful code base enlargement.
> I'd recommend to extend the docs, but the last word is yours ;)
>
> On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin <vl...@gmail.com> wrote:
>
>> Ivan, hi.
>>
>> I absolutely agree this particular description is not enough to see the
>> deactivation issue. I also vote for brief code.
>>
>> There are about 15 places in inner logic with this description. I
>> propose balance between code base size and comment completeness.
>>
>> Should we enlarge code even if we got several full descriptions?
>>
>>
>> 30.03.2020 20:02, Ivan Rakov пишет:
>>> Vladimir,
>>>
>>> @param forceDeactivation If {@code true}, cluster deactivation will be
>>>> forced.
>>> It's true that it's possible to infer semantics of forced deactivation
>> from
>>> other parts of API. I just wanted to highlight that exactly this
>>> description explains something that can be guessed by the parameter name.
>>> I suppose to shorten the lookup path and shed a light on deactivation
>>> semantics a bit:
>>>
>>>> @param forceDeactivation If {@code true}, cluster will be deactivated
>> even
>>>> if running in-memory caches are present. All data in the corresponding
>>>> caches will be vanished as a result.
>>> Does this make sense?
>>>
>>> On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
>>> wrote:
>>>
>>>> Ivan, hi.
>>>>
>>>>
>>>> 1) >>> Is it correct? If we are on the same page, let's proceed this way
>>>>
>>>> It is correct.
>>>>
>>>>
>>>> 2) - In many places in the code I can see the following javadoc
>>>>
>>>>>     @param forceDeactivation If {@code true}, cluster deactivation will
>> be
>>>> forced.
>>>>
>>>> In the internal params/flags. You can also find /@see
>>>> ClusterState#INACTIVE/ and full description with several public APIs (
>>>> like /Ignite.active(boolean)/ ):
>>>>
>>>> //
>>>>
>>>> /* <p>/
>>>>
>>>> //
>>>>
>>>> /* <b>NOTE:</b>/
>>>>
>>>> //
>>>>
>>>> /* Deactivation clears in-memory caches (without persistence) including
>>>> the system caches./
>>>>
>>>> Should be enough. Is not?
>>>>
>>>>
>>>> 27.03.2020 10:51, Ivan Rakov пишет:
>>>>> Vladimir, Igniters,
>>>>>
>>>>> Let's emphasize our final plan.
>>>>>
>>>>> We are going to add --force flags that will be necessary to pass for a
>>>>> deactivation if there are in-memory caches to:
>>>>> 1) Rest API (already implemented in [1])
>>>>> 2) Command line utility (already implemented in [1])
>>>>> 3) JMX bean (going to be implemented in [2])
>>>>> We are *not* going to change IgniteCluster or any other thick Java API,
>>>>> thus we are *not* going to merge [3].
>>>>> We plan to *fully rollback* [1] and [2] once cache data survival after
>>>>> activation-deactivation cycle will be implemented.
>>>>>
>>>>> Is it correct? If we are on the same page, let's proceed this way.
>>>>> I propose to:
>>>>> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
>>>>> without IEP and detailed design so far)
>>>>> - Describe in the issue description what exact parts of API should be
>>>>> removed under the issue scope.
>>>>>
>>>>> Also, a few questions on already merged [1]:
>>>>> - We have removed GridClientClusterState#state(ClusterState) from Java
>>>>> client API. Is it a legitimate thing to do? Don't we have to support
>> API
>>>>> compatibility for thin clients as well?
>>>>> - In many places in the code I can see the following javadoc
>>>>>
>>>>>>     @param forceDeactivation If {@code true}, cluster deactivation will
>>>> be forced.
>>>>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
>>>> describe
>>>>> in which cases deactivation won't happen unless it's forced and which
>>>>> impact forced deactivation will bring on the system.
>>>>>
>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
>>>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
>>>>>
>>>>> --
>>>>> Ivan
>>>>>
>>>>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com>
>>>> wrote:
>>>>>> Hi, Igniters.
>>>>>>
>>>>>> I'd like to remind you that cluster can be deactivated by user with 3
>>>>>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
>>>>>> not about control.sh. It suggests same approach regardless of the
>>>>>> utility user executes. The task touches *only* *API of the user
>> calls*,
>>>>>> not the internal APIs.
>>>>>>
>>>>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
>>>>>> account for control.sh are:
>>>>>>
>>>>>> -Various commands widely use “--yes” just to start. Even not dangerous
>>>>>> ones require “--yes” to begin. “--force” is dedicated for *harmless
>>>>>> actions*.
>>>>>>
>>>>>> -Checking of probable data erasure works after command start and
>>>>>> “--force” may not be required at all.
>>>>>>
>>>>>> -There are also JMX and REST. They have no “—yes” but should work
>> alike.
>>>>>>         To get the deactivation safe I propose to merge last ticket
>> with
>>>>>> the JMX fixes [2]. In future releases, I believe, we should estimate
>>>>>> jobs and fix memory erasure in general. For now, let’s prevent it.
>> WDYT?
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>>>>>>
>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>>>>>>
>>>>>>
>>>>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
>>>>>>> Hello Nikolay,
>>>>>>>
>>>>>>> I am talking about the interactive mode of the control utility, which
>>>>>>> requires explicit confirmation from the user.
>>>>>>> Please take a look at DeactivateCommand#prepareConfirmation and its
>>>>>> usages.
>>>>>>> It seems to me, this mode has the same aim as the forceDeactivation
>>>> flag.
>>>>>>> We can change the message returned by
>>>>>> DeactivateCommand#confirmationPrompt
>>>>>>> as follows:
>>>>>>>         "Warning: the command will deactivate the cluster nnn and
>> clear
>>>>>>> in-memory caches (without persistence) including system caches."
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> S.
>>>>>>>
>>>>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>>>>>>>
>>>>>>>> Hello, Slava.
>>>>>>>>
>>>>>>>> Are you talking about this commit [1] (sorry for commit message it’s
>>>> due
>>>>>>>> to the Github issue)?
>>>>>>>>
>>>>>>>> The message for this command for now
>>>>>>>>
>>>>>>>> «Deactivation stopped. Deactivation clears in-memory caches (without
>>>>>>>> persistence) including the system caches.»
>>>>>>>>
>>>>>>>> Is it clear enough?
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>>>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
>>>> slava.koptilin@gmail.com
>>>>>>>> написал(а):
>>>>>>>>> Hi Nikolay,
>>>>>>>>>
>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
>> command.
>>>>>>>>> I just checked and it seems that the deactivation command
>>>>>>>>> (control-utility.sh) already has a confirmation option.
>>>>>>>>> Perhaps, we need to clearly state the consequences of using this
>>>>>> command
>>>>>>>>> with in-memory caches.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> S.
>>>>>>>>>
>>>>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <nizhikov@apache.org
>>> :
>>>>>>>>>> Hello, Alexey.
>>>>>>>>>>
>>>>>>>>>> I just repeat our agreement to be on the same page
>>>>>>>>>>
>>>>>>>>>>> The confirmation should only present in the user-facing
>> interfaces.
>>>>>>>>>> 1. We should add —force flag to the command.sh deactivation
>> command.
>>>>>>>>>> 2. We should throw the exception if cluster has in-memory caches
>> and
>>>>>>>>>> —force=false.
>>>>>>>>>> 3. We shouldn’t change Java API for deactivation.
>>>>>>>>>>
>>>>>>>>>> Is it correct?
>>>>>>>>>>
>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
>> it
>>>>>>>>>> I think it because the command itself has a «DROP» word in it’s
>>>> name.
>>>>>>>>>> Which clearly explains what will happen on it’s execution.
>>>>>>>>>>
>>>>>>>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>>>>>>>> destructive operation.
>>>>>>>>>> That’s why we should warn user about it’s consequences.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>> написал(а):
>>>>>>>>>>> Igniters, Ivan, Nikolay,
>>>>>>>>>>>
>>>>>>>>>>> I am strongly against adding confirmation flags to any kind of
>>>> APIs,
>>>>>>>>>>> whether we change the deactivation behavior or not (even though I
>>>>>> agree
>>>>>>>>>>> that it makes sense to fix the deactivation to not clean up the
>>>>>>>> in-memory
>>>>>>>>>>> data). The confirmation should only present in the user-facing
>>>>>>>>>> interfaces.
>>>>>>>>>>> I cannot recall any software interface which has such a flag.
>> None
>>>> of
>>>>>>>> the
>>>>>>>>>>> syscalls which delete files (a very destructive operation) have
>>>> this
>>>>>>>>>> flag.
>>>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
>>>> it.
>>>>>>>> As I
>>>>>>>>>>> already mentioned, when used programmatically, most users will
>>>> likely
>>>>>>>>>>> simply pass 'true' as the new flag because they already know the
>>>>>>>>>> behavior.
>>>>>>>>>>> This is a clear sign of a bad design choice.
>>>>>>>>>>>
>>>>>>>>>>> On top of that, given that it is our intention to change the
>>>> behavior
>>>>>>>> of
>>>>>>>>>>> deactivation to not loose the in-memory data, it does not make
>>>> sense
>>>>>> to
>>>>>>>>>> me
>>>>>>>>>>> to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

I don't think that making javadocs more descriptive can be considered as
harmful code base enlargement.
I'd recommend to extend the docs, but the last word is yours ;)

On Tue, Mar 31, 2020 at 2:44 PM Vladimir Steshin <vl...@gmail.com> wrote:

> Ivan, hi.
>
> I absolutely agree this particular description is not enough to see the
> deactivation issue. I also vote for brief code.
>
> There are about 15 places in inner logic with this description. I
> propose balance between code base size and comment completeness.
>
> Should we enlarge code even if we got several full descriptions?
>
>
> 30.03.2020 20:02, Ivan Rakov пишет:
> > Vladimir,
> >
> > @param forceDeactivation If {@code true}, cluster deactivation will be
> >> forced.
> > It's true that it's possible to infer semantics of forced deactivation
> from
> > other parts of API. I just wanted to highlight that exactly this
> > description explains something that can be guessed by the parameter name.
> > I suppose to shorten the lookup path and shed a light on deactivation
> > semantics a bit:
> >
> >> @param forceDeactivation If {@code true}, cluster will be deactivated
> even
> >> if running in-memory caches are present. All data in the corresponding
> >> caches will be vanished as a result.
> > Does this make sense?
> >
> > On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
> > wrote:
> >
> >> Ivan, hi.
> >>
> >>
> >> 1) >>> Is it correct? If we are on the same page, let's proceed this way
> >>
> >> It is correct.
> >>
> >>
> >> 2) - In many places in the code I can see the following javadoc
> >>
> >>>    @param forceDeactivation If {@code true}, cluster deactivation will
> be
> >> forced.
> >>
> >> In the internal params/flags. You can also find /@see
> >> ClusterState#INACTIVE/ and full description with several public APIs (
> >> like /Ignite.active(boolean)/ ):
> >>
> >> //
> >>
> >> /* <p>/
> >>
> >> //
> >>
> >> /* <b>NOTE:</b>/
> >>
> >> //
> >>
> >> /* Deactivation clears in-memory caches (without persistence) including
> >> the system caches./
> >>
> >> Should be enough. Is not?
> >>
> >>
> >> 27.03.2020 10:51, Ivan Rakov пишет:
> >>> Vladimir, Igniters,
> >>>
> >>> Let's emphasize our final plan.
> >>>
> >>> We are going to add --force flags that will be necessary to pass for a
> >>> deactivation if there are in-memory caches to:
> >>> 1) Rest API (already implemented in [1])
> >>> 2) Command line utility (already implemented in [1])
> >>> 3) JMX bean (going to be implemented in [2])
> >>> We are *not* going to change IgniteCluster or any other thick Java API,
> >>> thus we are *not* going to merge [3].
> >>> We plan to *fully rollback* [1] and [2] once cache data survival after
> >>> activation-deactivation cycle will be implemented.
> >>>
> >>> Is it correct? If we are on the same page, let's proceed this way.
> >>> I propose to:
> >>> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
> >>> without IEP and detailed design so far)
> >>> - Describe in the issue description what exact parts of API should be
> >>> removed under the issue scope.
> >>>
> >>> Also, a few questions on already merged [1]:
> >>> - We have removed GridClientClusterState#state(ClusterState) from Java
> >>> client API. Is it a legitimate thing to do? Don't we have to support
> API
> >>> compatibility for thin clients as well?
> >>> - In many places in the code I can see the following javadoc
> >>>
> >>>>    @param forceDeactivation If {@code true}, cluster deactivation will
> >> be forced.
> >>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
> >> describe
> >>> in which cases deactivation won't happen unless it's forced and which
> >>> impact forced deactivation will bring on the system.
> >>>
> >>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> >>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> >>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
> >>>
> >>> --
> >>> Ivan
> >>>
> >>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com>
> >> wrote:
> >>>> Hi, Igniters.
> >>>>
> >>>> I'd like to remind you that cluster can be deactivated by user with 3
> >>>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
> >>>> not about control.sh. It suggests same approach regardless of the
> >>>> utility user executes. The task touches *only* *API of the user
> calls*,
> >>>> not the internal APIs.
> >>>>
> >>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
> >>>> account for control.sh are:
> >>>>
> >>>> -Various commands widely use “--yes” just to start. Even not dangerous
> >>>> ones require “--yes” to begin. “--force” is dedicated for *harmless
> >>>> actions*.
> >>>>
> >>>> -Checking of probable data erasure works after command start and
> >>>> “--force” may not be required at all.
> >>>>
> >>>> -There are also JMX and REST. They have no “—yes” but should work
> alike.
> >>>>
> >>>>        To get the deactivation safe I propose to merge last ticket
> with
> >>>> the JMX fixes [2]. In future releases, I believe, we should estimate
> >>>> jobs and fix memory erasure in general. For now, let’s prevent it.
> WDYT?
> >>>>
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>
> >>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
> >>>>
> >>>>
> >>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
> >>>>> Hello Nikolay,
> >>>>>
> >>>>> I am talking about the interactive mode of the control utility, which
> >>>>> requires explicit confirmation from the user.
> >>>>> Please take a look at DeactivateCommand#prepareConfirmation and its
> >>>> usages.
> >>>>> It seems to me, this mode has the same aim as the forceDeactivation
> >> flag.
> >>>>> We can change the message returned by
> >>>> DeactivateCommand#confirmationPrompt
> >>>>> as follows:
> >>>>>        "Warning: the command will deactivate the cluster nnn and
> clear
> >>>>> in-memory caches (without persistence) including system caches."
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>> Thanks,
> >>>>> S.
> >>>>>
> >>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
> >>>>>
> >>>>>> Hello, Slava.
> >>>>>>
> >>>>>> Are you talking about this commit [1] (sorry for commit message it’s
> >> due
> >>>>>> to the Github issue)?
> >>>>>>
> >>>>>> The message for this command for now
> >>>>>>
> >>>>>> «Deactivation stopped. Deactivation clears in-memory caches (without
> >>>>>> persistence) including the system caches.»
> >>>>>>
> >>>>>> Is it clear enough?
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
> >> slava.koptilin@gmail.com
> >>>>>> написал(а):
> >>>>>>> Hi Nikolay,
> >>>>>>>
> >>>>>>>> 1. We should add —force flag to the command.sh deactivation
> command.
> >>>>>>> I just checked and it seems that the deactivation command
> >>>>>>> (control-utility.sh) already has a confirmation option.
> >>>>>>> Perhaps, we need to clearly state the consequences of using this
> >>>> command
> >>>>>>> with in-memory caches.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> S.
> >>>>>>>
> >>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <nizhikov@apache.org
> >:
> >>>>>>>
> >>>>>>>> Hello, Alexey.
> >>>>>>>>
> >>>>>>>> I just repeat our agreement to be on the same page
> >>>>>>>>
> >>>>>>>>> The confirmation should only present in the user-facing
> interfaces.
> >>>>>>>> 1. We should add —force flag to the command.sh deactivation
> command.
> >>>>>>>> 2. We should throw the exception if cluster has in-memory caches
> and
> >>>>>>>> —force=false.
> >>>>>>>> 3. We shouldn’t change Java API for deactivation.
> >>>>>>>>
> >>>>>>>> Is it correct?
> >>>>>>>>
> >>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
> it
> >>>>>>>> I think it because the command itself has a «DROP» word in it’s
> >> name.
> >>>>>>>> Which clearly explains what will happen on it’s execution.
> >>>>>>>>
> >>>>>>>> On the opposite «deactivation» command doesn’t have any sign of
> >>>>>>>> destructive operation.
> >>>>>>>> That’s why we should warn user about it’s consequences.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>> написал(а):
> >>>>>>>>> Igniters, Ivan, Nikolay,
> >>>>>>>>>
> >>>>>>>>> I am strongly against adding confirmation flags to any kind of
> >> APIs,
> >>>>>>>>> whether we change the deactivation behavior or not (even though I
> >>>> agree
> >>>>>>>>> that it makes sense to fix the deactivation to not clean up the
> >>>>>> in-memory
> >>>>>>>>> data). The confirmation should only present in the user-facing
> >>>>>>>> interfaces.
> >>>>>>>>> I cannot recall any software interface which has such a flag.
> None
> >> of
> >>>>>> the
> >>>>>>>>> syscalls which delete files (a very destructive operation) have
> >> this
> >>>>>>>> flag.
> >>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
> >> it.
> >>>>>> As I
> >>>>>>>>> already mentioned, when used programmatically, most users will
> >> likely
> >>>>>>>>> simply pass 'true' as the new flag because they already know the
> >>>>>>>> behavior.
> >>>>>>>>> This is a clear sign of a bad design choice.
> >>>>>>>>>
> >>>>>>>>> On top of that, given that it is our intention to change the
> >> behavior
> >>>>>> of
> >>>>>>>>> deactivation to not loose the in-memory data, it does not make
> >> sense
> >>>> to
> >>>>>>>> me
> >>>>>>>>> to change the API.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Ivan, hi.

I absolutely agree this particular description is not enough to see the 
deactivation issue. I also vote for brief code.

There are about 15 places in inner logic with this description. I 
propose balance between code base size and comment completeness.

Should we enlarge code even if we got several full descriptions?


30.03.2020 20:02, Ivan Rakov пишет:
> Vladimir,
>
> @param forceDeactivation If {@code true}, cluster deactivation will be
>> forced.
> It's true that it's possible to infer semantics of forced deactivation from
> other parts of API. I just wanted to highlight that exactly this
> description explains something that can be guessed by the parameter name.
> I suppose to shorten the lookup path and shed a light on deactivation
> semantics a bit:
>
>> @param forceDeactivation If {@code true}, cluster will be deactivated even
>> if running in-memory caches are present. All data in the corresponding
>> caches will be vanished as a result.
> Does this make sense?
>
> On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
> wrote:
>
>> Ivan, hi.
>>
>>
>> 1) >>> Is it correct? If we are on the same page, let's proceed this way
>>
>> It is correct.
>>
>>
>> 2) - In many places in the code I can see the following javadoc
>>
>>>    @param forceDeactivation If {@code true}, cluster deactivation will be
>> forced.
>>
>> In the internal params/flags. You can also find /@see
>> ClusterState#INACTIVE/ and full description with several public APIs (
>> like /Ignite.active(boolean)/ ):
>>
>> //
>>
>> /* <p>/
>>
>> //
>>
>> /* <b>NOTE:</b>/
>>
>> //
>>
>> /* Deactivation clears in-memory caches (without persistence) including
>> the system caches./
>>
>> Should be enough. Is not?
>>
>>
>> 27.03.2020 10:51, Ivan Rakov пишет:
>>> Vladimir, Igniters,
>>>
>>> Let's emphasize our final plan.
>>>
>>> We are going to add --force flags that will be necessary to pass for a
>>> deactivation if there are in-memory caches to:
>>> 1) Rest API (already implemented in [1])
>>> 2) Command line utility (already implemented in [1])
>>> 3) JMX bean (going to be implemented in [2])
>>> We are *not* going to change IgniteCluster or any other thick Java API,
>>> thus we are *not* going to merge [3].
>>> We plan to *fully rollback* [1] and [2] once cache data survival after
>>> activation-deactivation cycle will be implemented.
>>>
>>> Is it correct? If we are on the same page, let's proceed this way.
>>> I propose to:
>>> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
>>> without IEP and detailed design so far)
>>> - Describe in the issue description what exact parts of API should be
>>> removed under the issue scope.
>>>
>>> Also, a few questions on already merged [1]:
>>> - We have removed GridClientClusterState#state(ClusterState) from Java
>>> client API. Is it a legitimate thing to do? Don't we have to support API
>>> compatibility for thin clients as well?
>>> - In many places in the code I can see the following javadoc
>>>
>>>>    @param forceDeactivation If {@code true}, cluster deactivation will
>> be forced.
>>>> As for me, this javadoc doesn't clarify anything. I'd suggest to
>> describe
>>> in which cases deactivation won't happen unless it's forced and which
>>> impact forced deactivation will bring on the system.
>>>
>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
>>>
>>> --
>>> Ivan
>>>
>>> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com>
>> wrote:
>>>> Hi, Igniters.
>>>>
>>>> I'd like to remind you that cluster can be deactivated by user with 3
>>>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
>>>> not about control.sh. It suggests same approach regardless of the
>>>> utility user executes. The task touches *only* *API of the user calls*,
>>>> not the internal APIs.
>>>>
>>>> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
>>>> account for control.sh are:
>>>>
>>>> -Various commands widely use “--yes” just to start. Even not dangerous
>>>> ones require “--yes” to begin. “--force” is dedicated for *harmless
>>>> actions*.
>>>>
>>>> -Checking of probable data erasure works after command start and
>>>> “--force” may not be required at all.
>>>>
>>>> -There are also JMX and REST. They have no “—yes” but should work alike.
>>>>
>>>>        To get the deactivation safe I propose to merge last ticket with
>>>> the JMX fixes [2]. In future releases, I believe, we should estimate
>>>> jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>>>>
>>>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>>>>
>>>>
>>>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
>>>>> Hello Nikolay,
>>>>>
>>>>> I am talking about the interactive mode of the control utility, which
>>>>> requires explicit confirmation from the user.
>>>>> Please take a look at DeactivateCommand#prepareConfirmation and its
>>>> usages.
>>>>> It seems to me, this mode has the same aim as the forceDeactivation
>> flag.
>>>>> We can change the message returned by
>>>> DeactivateCommand#confirmationPrompt
>>>>> as follows:
>>>>>        "Warning: the command will deactivate the cluster nnn and clear
>>>>> in-memory caches (without persistence) including system caches."
>>>>>
>>>>> What do you think?
>>>>>
>>>>> Thanks,
>>>>> S.
>>>>>
>>>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>>>>>
>>>>>> Hello, Slava.
>>>>>>
>>>>>> Are you talking about this commit [1] (sorry for commit message it’s
>> due
>>>>>> to the Github issue)?
>>>>>>
>>>>>> The message for this command for now
>>>>>>
>>>>>> «Deactivation stopped. Deactivation clears in-memory caches (without
>>>>>> persistence) including the system caches.»
>>>>>>
>>>>>> Is it clear enough?
>>>>>>
>>>>>> [1]
>>>>>>
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
>> slava.koptilin@gmail.com
>>>>>> написал(а):
>>>>>>> Hi Nikolay,
>>>>>>>
>>>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>>>> I just checked and it seems that the deactivation command
>>>>>>> (control-utility.sh) already has a confirmation option.
>>>>>>> Perhaps, we need to clearly state the consequences of using this
>>>> command
>>>>>>> with in-memory caches.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> S.
>>>>>>>
>>>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
>>>>>>>
>>>>>>>> Hello, Alexey.
>>>>>>>>
>>>>>>>> I just repeat our agreement to be on the same page
>>>>>>>>
>>>>>>>>> The confirmation should only present in the user-facing interfaces.
>>>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>>>>> 2. We should throw the exception if cluster has in-memory caches and
>>>>>>>> —force=false.
>>>>>>>> 3. We shouldn’t change Java API for deactivation.
>>>>>>>>
>>>>>>>> Is it correct?
>>>>>>>>
>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
>>>>>>>> I think it because the command itself has a «DROP» word in it’s
>> name.
>>>>>>>> Which clearly explains what will happen on it’s execution.
>>>>>>>>
>>>>>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>>>>>> destructive operation.
>>>>>>>> That’s why we should warn user about it’s consequences.
>>>>>>>>
>>>>>>>>
>>>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>> написал(а):
>>>>>>>>> Igniters, Ivan, Nikolay,
>>>>>>>>>
>>>>>>>>> I am strongly against adding confirmation flags to any kind of
>> APIs,
>>>>>>>>> whether we change the deactivation behavior or not (even though I
>>>> agree
>>>>>>>>> that it makes sense to fix the deactivation to not clean up the
>>>>>> in-memory
>>>>>>>>> data). The confirmation should only present in the user-facing
>>>>>>>> interfaces.
>>>>>>>>> I cannot recall any software interface which has such a flag. None
>> of
>>>>>> the
>>>>>>>>> syscalls which delete files (a very destructive operation) have
>> this
>>>>>>>> flag.
>>>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
>> it.
>>>>>> As I
>>>>>>>>> already mentioned, when used programmatically, most users will
>> likely
>>>>>>>>> simply pass 'true' as the new flag because they already know the
>>>>>>>> behavior.
>>>>>>>>> This is a clear sign of a bad design choice.
>>>>>>>>>
>>>>>>>>> On top of that, given that it is our intention to change the
>> behavior
>>>>>> of
>>>>>>>>> deactivation to not loose the in-memory data, it does not make
>> sense
>>>> to
>>>>>>>> me
>>>>>>>>> to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

Vladimir,

@param forceDeactivation If {@code true}, cluster deactivation will be
> forced.

It's true that it's possible to infer semantics of forced deactivation from
other parts of API. I just wanted to highlight that exactly this
description explains something that can be guessed by the parameter name.
I suppose to shorten the lookup path and shed a light on deactivation
semantics a bit:

> @param forceDeactivation If {@code true}, cluster will be deactivated even
> if running in-memory caches are present. All data in the corresponding
> caches will be vanished as a result.

Does this make sense?

On Fri, Mar 27, 2020 at 12:00 PM Vladimir Steshin <vl...@gmail.com>
wrote:

> Ivan, hi.
>
>
> 1) >>> Is it correct? If we are on the same page, let's proceed this way
>
> It is correct.
>
>
> 2) - In many places in the code I can see the following javadoc
>
> >   @param forceDeactivation If {@code true}, cluster deactivation will be
> forced.
>
> In the internal params/flags. You can also find /@see
> ClusterState#INACTIVE/ and full description with several public APIs (
> like /Ignite.active(boolean)/ ):
>
> //
>
> /* <p>/
>
> //
>
> /* <b>NOTE:</b>/
>
> //
>
> /* Deactivation clears in-memory caches (without persistence) including
> the system caches./
>
> Should be enough. Is not?
>
>
> 27.03.2020 10:51, Ivan Rakov пишет:
> > Vladimir, Igniters,
> >
> > Let's emphasize our final plan.
> >
> > We are going to add --force flags that will be necessary to pass for a
> > deactivation if there are in-memory caches to:
> > 1) Rest API (already implemented in [1])
> > 2) Command line utility (already implemented in [1])
> > 3) JMX bean (going to be implemented in [2])
> > We are *not* going to change IgniteCluster or any other thick Java API,
> > thus we are *not* going to merge [3].
> > We plan to *fully rollback* [1] and [2] once cache data survival after
> > activation-deactivation cycle will be implemented.
> >
> > Is it correct? If we are on the same page, let's proceed this way.
> > I propose to:
> > - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
> > without IEP and detailed design so far)
> > - Describe in the issue description what exact parts of API should be
> > removed under the issue scope.
> >
> > Also, a few questions on already merged [1]:
> > - We have removed GridClientClusterState#state(ClusterState) from Java
> > client API. Is it a legitimate thing to do? Don't we have to support API
> > compatibility for thin clients as well?
> > - In many places in the code I can see the following javadoc
> >
> >>   @param forceDeactivation If {@code true}, cluster deactivation will
> be forced.
> >>
> >> As for me, this javadoc doesn't clarify anything. I'd suggest to
> describe
> > in which cases deactivation won't happen unless it's forced and which
> > impact forced deactivation will bring on the system.
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> > [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> > [3]: https://issues.apache.org/jira/browse/IGNITE-12614
> >
> > --
> > Ivan
> >
> > On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com>
> wrote:
> >
> >> Hi, Igniters.
> >>
> >> I'd like to remind you that cluster can be deactivated by user with 3
> >> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
> >> not about control.sh. It suggests same approach regardless of the
> >> utility user executes. The task touches *only* *API of the user calls*,
> >> not the internal APIs.
> >>
> >> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
> >> account for control.sh are:
> >>
> >> -Various commands widely use “--yes” just to start. Even not dangerous
> >> ones require “--yes” to begin. “--force” is dedicated for *harmless
> >> actions*.
> >>
> >> -Checking of probable data erasure works after command start and
> >> “--force” may not be required at all.
> >>
> >> -There are also JMX and REST. They have no “—yes” but should work alike.
> >>
> >>       To get the deactivation safe I propose to merge last ticket with
> >> the JMX fixes [2]. In future releases, I believe, we should estimate
> >> jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >>
> >> [2] https://issues.apache.org/jira/browse/IGNITE-12779
> >>
> >>
> >> 24.03.2020 15:55, Вячеслав Коптилин пишет:
> >>> Hello Nikolay,
> >>>
> >>> I am talking about the interactive mode of the control utility, which
> >>> requires explicit confirmation from the user.
> >>> Please take a look at DeactivateCommand#prepareConfirmation and its
> >> usages.
> >>> It seems to me, this mode has the same aim as the forceDeactivation
> flag.
> >>> We can change the message returned by
> >> DeactivateCommand#confirmationPrompt
> >>> as follows:
> >>>       "Warning: the command will deactivate the cluster nnn and clear
> >>> in-memory caches (without persistence) including system caches."
> >>>
> >>> What do you think?
> >>>
> >>> Thanks,
> >>> S.
> >>>
> >>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
> >>>
> >>>> Hello, Slava.
> >>>>
> >>>> Are you talking about this commit [1] (sorry for commit message it’s
> due
> >>>> to the Github issue)?
> >>>>
> >>>> The message for this command for now
> >>>>
> >>>> «Deactivation stopped. Deactivation clears in-memory caches (without
> >>>> persistence) including the system caches.»
> >>>>
> >>>> Is it clear enough?
> >>>>
> >>>> [1]
> >>>>
> >>
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >>>>
> >>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <
> slava.koptilin@gmail.com
> >>>> написал(а):
> >>>>> Hi Nikolay,
> >>>>>
> >>>>>> 1. We should add —force flag to the command.sh deactivation command.
> >>>>> I just checked and it seems that the deactivation command
> >>>>> (control-utility.sh) already has a confirmation option.
> >>>>> Perhaps, we need to clearly state the consequences of using this
> >> command
> >>>>> with in-memory caches.
> >>>>>
> >>>>> Thanks,
> >>>>> S.
> >>>>>
> >>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
> >>>>>
> >>>>>> Hello, Alexey.
> >>>>>>
> >>>>>> I just repeat our agreement to be on the same page
> >>>>>>
> >>>>>>> The confirmation should only present in the user-facing interfaces.
> >>>>>> 1. We should add —force flag to the command.sh deactivation command.
> >>>>>> 2. We should throw the exception if cluster has in-memory caches and
> >>>>>> —force=false.
> >>>>>> 3. We shouldn’t change Java API for deactivation.
> >>>>>>
> >>>>>> Is it correct?
> >>>>>>
> >>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
> >>>>>> I think it because the command itself has a «DROP» word in it’s
> name.
> >>>>>> Which clearly explains what will happen on it’s execution.
> >>>>>>
> >>>>>> On the opposite «deactivation» command doesn’t have any sign of
> >>>>>> destructive operation.
> >>>>>> That’s why we should warn user about it’s consequences.
> >>>>>>
> >>>>>>
> >>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> >>>> alexey.goncharuk@gmail.com>
> >>>>>> написал(а):
> >>>>>>> Igniters, Ivan, Nikolay,
> >>>>>>>
> >>>>>>> I am strongly against adding confirmation flags to any kind of
> APIs,
> >>>>>>> whether we change the deactivation behavior or not (even though I
> >> agree
> >>>>>>> that it makes sense to fix the deactivation to not clean up the
> >>>> in-memory
> >>>>>>> data). The confirmation should only present in the user-facing
> >>>>>> interfaces.
> >>>>>>> I cannot recall any software interface which has such a flag. None
> of
> >>>> the
> >>>>>>> syscalls which delete files (a very destructive operation) have
> this
> >>>>>> flag.
> >>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in
> it.
> >>>> As I
> >>>>>>> already mentioned, when used programmatically, most users will
> likely
> >>>>>>> simply pass 'true' as the new flag because they already know the
> >>>>>> behavior.
> >>>>>>> This is a clear sign of a bad design choice.
> >>>>>>>
> >>>>>>> On top of that, given that it is our intention to change the
> behavior
> >>>> of
> >>>>>>> deactivation to not loose the in-memory data, it does not make
> sense
> >> to
> >>>>>> me
> >>>>>>> to change the API.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Ivan, hi.


1) >>> Is it correct? If we are on the same page, let's proceed this way

It is correct.


2) - In many places in the code I can see the following javadoc

>   @param forceDeactivation If {@code true}, cluster deactivation will be forced.

In the internal params/flags. You can also find /@see 
ClusterState#INACTIVE/ and full description with several public APIs ( 
like /Ignite.active(boolean)/ ):

//

/* <p>/

//

/* <b>NOTE:</b>/

//

/* Deactivation clears in-memory caches (without persistence) including 
the system caches./

Should be enough. Is not?


27.03.2020 10:51, Ivan Rakov пишет:
> Vladimir, Igniters,
>
> Let's emphasize our final plan.
>
> We are going to add --force flags that will be necessary to pass for a
> deactivation if there are in-memory caches to:
> 1) Rest API (already implemented in [1])
> 2) Command line utility (already implemented in [1])
> 3) JMX bean (going to be implemented in [2])
> We are *not* going to change IgniteCluster or any other thick Java API,
> thus we are *not* going to merge [3].
> We plan to *fully rollback* [1] and [2] once cache data survival after
> activation-deactivation cycle will be implemented.
>
> Is it correct? If we are on the same page, let's proceed this way.
> I propose to:
> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
> without IEP and detailed design so far)
> - Describe in the issue description what exact parts of API should be
> removed under the issue scope.
>
> Also, a few questions on already merged [1]:
> - We have removed GridClientClusterState#state(ClusterState) from Java
> client API. Is it a legitimate thing to do? Don't we have to support API
> compatibility for thin clients as well?
> - In many places in the code I can see the following javadoc
>
>>   @param forceDeactivation If {@code true}, cluster deactivation will be forced.
>>
>> As for me, this javadoc doesn't clarify anything. I'd suggest to describe
> in which cases deactivation won't happen unless it's forced and which
> impact forced deactivation will bring on the system.
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
>
> --
> Ivan
>
> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com> wrote:
>
>> Hi, Igniters.
>>
>> I'd like to remind you that cluster can be deactivated by user with 3
>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
>> not about control.sh. It suggests same approach regardless of the
>> utility user executes. The task touches *only* *API of the user calls*,
>> not the internal APIs.
>>
>> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
>> account for control.sh are:
>>
>> -Various commands widely use “--yes” just to start. Even not dangerous
>> ones require “--yes” to begin. “--force” is dedicated for *harmless
>> actions*.
>>
>> -Checking of probable data erasure works after command start and
>> “--force” may not be required at all.
>>
>> -There are also JMX and REST. They have no “—yes” but should work alike.
>>
>>       To get the deactivation safe I propose to merge last ticket with
>> the JMX fixes [2]. In future releases, I believe, we should estimate
>> jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>>
>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>>
>>
>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
>>> Hello Nikolay,
>>>
>>> I am talking about the interactive mode of the control utility, which
>>> requires explicit confirmation from the user.
>>> Please take a look at DeactivateCommand#prepareConfirmation and its
>> usages.
>>> It seems to me, this mode has the same aim as the forceDeactivation flag.
>>> We can change the message returned by
>> DeactivateCommand#confirmationPrompt
>>> as follows:
>>>       "Warning: the command will deactivate the cluster nnn and clear
>>> in-memory caches (without persistence) including system caches."
>>>
>>> What do you think?
>>>
>>> Thanks,
>>> S.
>>>
>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>>>
>>>> Hello, Slava.
>>>>
>>>> Are you talking about this commit [1] (sorry for commit message it’s due
>>>> to the Github issue)?
>>>>
>>>> The message for this command for now
>>>>
>>>> «Deactivation stopped. Deactivation clears in-memory caches (without
>>>> persistence) including the system caches.»
>>>>
>>>> Is it clear enough?
>>>>
>>>> [1]
>>>>
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>>>
>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <slava.koptilin@gmail.com
>>>> написал(а):
>>>>> Hi Nikolay,
>>>>>
>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>> I just checked and it seems that the deactivation command
>>>>> (control-utility.sh) already has a confirmation option.
>>>>> Perhaps, we need to clearly state the consequences of using this
>> command
>>>>> with in-memory caches.
>>>>>
>>>>> Thanks,
>>>>> S.
>>>>>
>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
>>>>>
>>>>>> Hello, Alexey.
>>>>>>
>>>>>> I just repeat our agreement to be on the same page
>>>>>>
>>>>>>> The confirmation should only present in the user-facing interfaces.
>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>>> 2. We should throw the exception if cluster has in-memory caches and
>>>>>> —force=false.
>>>>>> 3. We shouldn’t change Java API for deactivation.
>>>>>>
>>>>>> Is it correct?
>>>>>>
>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
>>>>>> I think it because the command itself has a «DROP» word in it’s name.
>>>>>> Which clearly explains what will happen on it’s execution.
>>>>>>
>>>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>>>> destructive operation.
>>>>>> That’s why we should warn user about it’s consequences.
>>>>>>
>>>>>>
>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>>>> alexey.goncharuk@gmail.com>
>>>>>> написал(а):
>>>>>>> Igniters, Ivan, Nikolay,
>>>>>>>
>>>>>>> I am strongly against adding confirmation flags to any kind of APIs,
>>>>>>> whether we change the deactivation behavior or not (even though I
>> agree
>>>>>>> that it makes sense to fix the deactivation to not clean up the
>>>> in-memory
>>>>>>> data). The confirmation should only present in the user-facing
>>>>>> interfaces.
>>>>>>> I cannot recall any software interface which has such a flag. None of
>>>> the
>>>>>>> syscalls which delete files (a very destructive operation) have this
>>>>>> flag.
>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it.
>>>> As I
>>>>>>> already mentioned, when used programmatically, most users will likely
>>>>>>> simply pass 'true' as the new flag because they already know the
>>>>>> behavior.
>>>>>>> This is a clear sign of a bad design choice.
>>>>>>>
>>>>>>> On top of that, given that it is our intention to change the behavior
>>>> of
>>>>>>> deactivation to not loose the in-memory data, it does not make sense
>> to
>>>>>> me
>>>>>>> to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Ivan.

> Is it correct? If we are on the same page, let's proceed this way.

+1 from me for your plan.

> We have removed GridClientClusterState#state(ClusterState) from Java client API. Is it a legitimate thing to do? Don't we have to support API

There is no changes in Public API except discussed in this thread.
GridClientClusterState is in internal package so it’s an internal class.

org/apache/ignite/internal/client/GridClientClusterState.java 

> I'd suggest to describe in which cases deactivation won't happen unless it's forced and which impact forced deactivation will bring on the system.

OK for me.
Feel free to suggest your statement.

> 27 марта 2020 г., в 10:51, Ivan Rakov <iv...@gmail.com> написал(а):
> 
> Vladimir, Igniters,
> 
> Let's emphasize our final plan.
> 
> We are going to add --force flags that will be necessary to pass for a
> deactivation if there are in-memory caches to:
> 1) Rest API (already implemented in [1])
> 2) Command line utility (already implemented in [1])
> 3) JMX bean (going to be implemented in [2])
> We are *not* going to change IgniteCluster or any other thick Java API,
> thus we are *not* going to merge [3].
> We plan to *fully rollback* [1] and [2] once cache data survival after
> activation-deactivation cycle will be implemented.
> 
> Is it correct? If we are on the same page, let's proceed this way.
> I propose to:
> - Create a JIRA issue for in-memory-data-safe deactivation (possibly,
> without IEP and detailed design so far)
> - Describe in the issue description what exact parts of API should be
> removed under the issue scope.
> 
> Also, a few questions on already merged [1]:
> - We have removed GridClientClusterState#state(ClusterState) from Java
> client API. Is it a legitimate thing to do? Don't we have to support API
> compatibility for thin clients as well?
> - In many places in the code I can see the following javadoc
> 
>> @param forceDeactivation If {@code true}, cluster deactivation will be forced.
>> 
>> As for me, this javadoc doesn't clarify anything. I'd suggest to describe
> in which cases deactivation won't happen unless it's forced and which
> impact forced deactivation will bring on the system.
> 
> [1]: https://issues.apache.org/jira/browse/IGNITE-12701
> [2]: https://issues.apache.org/jira/browse/IGNITE-12779
> [3]: https://issues.apache.org/jira/browse/IGNITE-12614
> 
> --
> Ivan
> 
> On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com> wrote:
> 
>> Hi, Igniters.
>> 
>> I'd like to remind you that cluster can be deactivated by user with 3
>> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
>> not about control.sh. It suggests same approach regardless of the
>> utility user executes. The task touches *only* *API of the user calls*,
>> not the internal APIs.
>> 
>> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
>> account for control.sh are:
>> 
>> -Various commands widely use “--yes” just to start. Even not dangerous
>> ones require “--yes” to begin. “--force” is dedicated for *harmless
>> actions*.
>> 
>> -Checking of probable data erasure works after command start and
>> “--force” may not be required at all.
>> 
>> -There are also JMX and REST. They have no “—yes” but should work alike.
>> 
>>     To get the deactivation safe I propose to merge last ticket with
>> the JMX fixes [2]. In future releases, I believe, we should estimate
>> jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>> 
>> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>> 
>> 
>> 24.03.2020 15:55, Вячеслав Коптилин пишет:
>>> Hello Nikolay,
>>> 
>>> I am talking about the interactive mode of the control utility, which
>>> requires explicit confirmation from the user.
>>> Please take a look at DeactivateCommand#prepareConfirmation and its
>> usages.
>>> It seems to me, this mode has the same aim as the forceDeactivation flag.
>>> We can change the message returned by
>> DeactivateCommand#confirmationPrompt
>>> as follows:
>>>     "Warning: the command will deactivate the cluster nnn and clear
>>> in-memory caches (without persistence) including system caches."
>>> 
>>> What do you think?
>>> 
>>> Thanks,
>>> S.
>>> 
>>> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>>> 
>>>> Hello, Slava.
>>>> 
>>>> Are you talking about this commit [1] (sorry for commit message it’s due
>>>> to the Github issue)?
>>>> 
>>>> The message for this command for now
>>>> 
>>>> «Deactivation stopped. Deactivation clears in-memory caches (without
>>>> persistence) including the system caches.»
>>>> 
>>>> Is it clear enough?
>>>> 
>>>> [1]
>>>> 
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>>> 
>>>> 
>>>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <slava.koptilin@gmail.com
>>> 
>>>> написал(а):
>>>>> Hi Nikolay,
>>>>> 
>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>> I just checked and it seems that the deactivation command
>>>>> (control-utility.sh) already has a confirmation option.
>>>>> Perhaps, we need to clearly state the consequences of using this
>> command
>>>>> with in-memory caches.
>>>>> 
>>>>> Thanks,
>>>>> S.
>>>>> 
>>>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
>>>>> 
>>>>>> Hello, Alexey.
>>>>>> 
>>>>>> I just repeat our agreement to be on the same page
>>>>>> 
>>>>>>> The confirmation should only present in the user-facing interfaces.
>>>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>>>> 2. We should throw the exception if cluster has in-memory caches and
>>>>>> —force=false.
>>>>>> 3. We shouldn’t change Java API for deactivation.
>>>>>> 
>>>>>> Is it correct?
>>>>>> 
>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
>>>>>> I think it because the command itself has a «DROP» word in it’s name.
>>>>>> Which clearly explains what will happen on it’s execution.
>>>>>> 
>>>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>>>> destructive operation.
>>>>>> That’s why we should warn user about it’s consequences.
>>>>>> 
>>>>>> 
>>>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>>>> alexey.goncharuk@gmail.com>
>>>>>> написал(а):
>>>>>>> Igniters, Ivan, Nikolay,
>>>>>>> 
>>>>>>> I am strongly against adding confirmation flags to any kind of APIs,
>>>>>>> whether we change the deactivation behavior or not (even though I
>> agree
>>>>>>> that it makes sense to fix the deactivation to not clean up the
>>>> in-memory
>>>>>>> data). The confirmation should only present in the user-facing
>>>>>> interfaces.
>>>>>>> I cannot recall any software interface which has such a flag. None of
>>>> the
>>>>>>> syscalls which delete files (a very destructive operation) have this
>>>>>> flag.
>>>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it.
>>>> As I
>>>>>>> already mentioned, when used programmatically, most users will likely
>>>>>>> simply pass 'true' as the new flag because they already know the
>>>>>> behavior.
>>>>>>> This is a clear sign of a bad design choice.
>>>>>>> 
>>>>>>> On top of that, given that it is our intention to change the behavior
>>>> of
>>>>>>> deactivation to not loose the in-memory data, it does not make sense
>> to
>>>>>> me
>>>>>>> to change the API.
>>>>>> 
>>>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

Vladimir, Igniters,

Let's emphasize our final plan.

We are going to add --force flags that will be necessary to pass for a
deactivation if there are in-memory caches to:
1) Rest API (already implemented in [1])
2) Command line utility (already implemented in [1])
3) JMX bean (going to be implemented in [2])
We are *not* going to change IgniteCluster or any other thick Java API,
thus we are *not* going to merge [3].
We plan to *fully rollback* [1] and [2] once cache data survival after
activation-deactivation cycle will be implemented.

Is it correct? If we are on the same page, let's proceed this way.
I propose to:
- Create a JIRA issue for in-memory-data-safe deactivation (possibly,
without IEP and detailed design so far)
- Describe in the issue description what exact parts of API should be
removed under the issue scope.

Also, a few questions on already merged [1]:
- We have removed GridClientClusterState#state(ClusterState) from Java
client API. Is it a legitimate thing to do? Don't we have to support API
compatibility for thin clients as well?
- In many places in the code I can see the following javadoc

>  @param forceDeactivation If {@code true}, cluster deactivation will be forced.
>
> As for me, this javadoc doesn't clarify anything. I'd suggest to describe
in which cases deactivation won't happen unless it's forced and which
impact forced deactivation will bring on the system.

[1]: https://issues.apache.org/jira/browse/IGNITE-12701
[2]: https://issues.apache.org/jira/browse/IGNITE-12779
[3]: https://issues.apache.org/jira/browse/IGNITE-12614

--
Ivan

On Tue, Mar 24, 2020 at 7:18 PM Vladimir Steshin <vl...@gmail.com> wrote:

> Hi, Igniters.
>
> I'd like to remind you that cluster can be deactivated by user with 3
> utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is
> not about control.sh. It suggests same approach regardless of the
> utility user executes. The task touches *only* *API of the user calls*,
> not the internal APIs.
>
> The reasons why flag “--yes” and confirmation prompt hasn’t taken into
> account for control.sh are:
>
> -Various commands widely use “--yes” just to start. Even not dangerous
> ones require “--yes” to begin. “--force” is dedicated for *harmless
> actions*.
>
> -Checking of probable data erasure works after command start and
> “--force” may not be required at all.
>
> -There are also JMX and REST. They have no “—yes” but should work alike.
>
>      To get the deactivation safe I propose to merge last ticket with
> the JMX fixes [2]. In future releases, I believe, we should estimate
> jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>
> [2] https://issues.apache.org/jira/browse/IGNITE-12779
>
>
> 24.03.2020 15:55, Вячеслав Коптилин пишет:
> > Hello Nikolay,
> >
> > I am talking about the interactive mode of the control utility, which
> > requires explicit confirmation from the user.
> > Please take a look at DeactivateCommand#prepareConfirmation and its
> usages.
> > It seems to me, this mode has the same aim as the forceDeactivation flag.
> > We can change the message returned by
> DeactivateCommand#confirmationPrompt
> > as follows:
> >      "Warning: the command will deactivate the cluster nnn and clear
> > in-memory caches (without persistence) including system caches."
> >
> > What do you think?
> >
> > Thanks,
> > S.
> >
> > вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
> >
> >> Hello, Slava.
> >>
> >> Are you talking about this commit [1] (sorry for commit message it’s due
> >> to the Github issue)?
> >>
> >> The message for this command for now
> >>
> >> «Deactivation stopped. Deactivation clears in-memory caches (without
> >> persistence) including the system caches.»
> >>
> >> Is it clear enough?
> >>
> >> [1]
> >>
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
> >>
> >>
> >>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <slava.koptilin@gmail.com
> >
> >> написал(а):
> >>> Hi Nikolay,
> >>>
> >>>> 1. We should add —force flag to the command.sh deactivation command.
> >>> I just checked and it seems that the deactivation command
> >>> (control-utility.sh) already has a confirmation option.
> >>> Perhaps, we need to clearly state the consequences of using this
> command
> >>> with in-memory caches.
> >>>
> >>> Thanks,
> >>> S.
> >>>
> >>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
> >>>
> >>>> Hello, Alexey.
> >>>>
> >>>> I just repeat our agreement to be on the same page
> >>>>
> >>>>> The confirmation should only present in the user-facing interfaces.
> >>>> 1. We should add —force flag to the command.sh deactivation command.
> >>>> 2. We should throw the exception if cluster has in-memory caches and
> >>>> —force=false.
> >>>> 3. We shouldn’t change Java API for deactivation.
> >>>>
> >>>> Is it correct?
> >>>>
> >>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
> >>>> I think it because the command itself has a «DROP» word in it’s name.
> >>>> Which clearly explains what will happen on it’s execution.
> >>>>
> >>>> On the opposite «deactivation» command doesn’t have any sign of
> >>>> destructive operation.
> >>>> That’s why we should warn user about it’s consequences.
> >>>>
> >>>>
> >>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com>
> >>>> написал(а):
> >>>>> Igniters, Ivan, Nikolay,
> >>>>>
> >>>>> I am strongly against adding confirmation flags to any kind of APIs,
> >>>>> whether we change the deactivation behavior or not (even though I
> agree
> >>>>> that it makes sense to fix the deactivation to not clean up the
> >> in-memory
> >>>>> data). The confirmation should only present in the user-facing
> >>>> interfaces.
> >>>>> I cannot recall any software interface which has such a flag. None of
> >> the
> >>>>> syscalls which delete files (a very destructive operation) have this
> >>>> flag.
> >>>>> The DROP TABLE command does not have a "yes I am sure" clause in it.
> >> As I
> >>>>> already mentioned, when used programmatically, most users will likely
> >>>>> simply pass 'true' as the new flag because they already know the
> >>>> behavior.
> >>>>> This is a clear sign of a bad design choice.
> >>>>>
> >>>>> On top of that, given that it is our intention to change the behavior
> >> of
> >>>>> deactivation to not loose the in-memory data, it does not make sense
> to
> >>>> me
> >>>>> to change the API.
> >>>>
> >>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hi, Igniters.

I'd like to remind you that cluster can be deactivated by user with 3 
utilities: control.sh, *JMX and the REST*. Proposed in [1] solution is 
not about control.sh. It suggests same approach regardless of the 
utility user executes. The task touches *only* *API of the user calls*, 
not the internal APIs.

The reasons why flag “--yes” and confirmation prompt hasn’t taken into 
account for control.sh are:

-Various commands widely use “--yes” just to start. Even not dangerous 
ones require “--yes” to begin. “--force” is dedicated for *harmless 
actions*.

-Checking of probable data erasure works after command start and 
“--force” may not be required at all.

-There are also JMX and REST. They have no “—yes” but should work alike.

     To get the deactivation safe I propose to merge last ticket with 
the JMX fixes [2]. In future releases, I believe, we should estimate 
jobs and fix memory erasure in general. For now, let’s prevent it. WDYT?


[1] https://issues.apache.org/jira/browse/IGNITE-12614

[2] https://issues.apache.org/jira/browse/IGNITE-12779


24.03.2020 15:55, Вячеслав Коптилин пишет:
> Hello Nikolay,
>
> I am talking about the interactive mode of the control utility, which
> requires explicit confirmation from the user.
> Please take a look at DeactivateCommand#prepareConfirmation and its usages.
> It seems to me, this mode has the same aim as the forceDeactivation flag.
> We can change the message returned by DeactivateCommand#confirmationPrompt
> as follows:
>      "Warning: the command will deactivate the cluster nnn and clear
> in-memory caches (without persistence) including system caches."
>
> What do you think?
>
> Thanks,
> S.
>
> вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:
>
>> Hello, Slava.
>>
>> Are you talking about this commit [1] (sorry for commit message it’s due
>> to the Github issue)?
>>
>> The message for this command for now
>>
>> «Deactivation stopped. Deactivation clears in-memory caches (without
>> persistence) including the system caches.»
>>
>> Is it clear enough?
>>
>> [1]
>> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>>
>>
>>> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <sl...@gmail.com>
>> написал(а):
>>> Hi Nikolay,
>>>
>>>> 1. We should add —force flag to the command.sh deactivation command.
>>> I just checked and it seems that the deactivation command
>>> (control-utility.sh) already has a confirmation option.
>>> Perhaps, we need to clearly state the consequences of using this command
>>> with in-memory caches.
>>>
>>> Thanks,
>>> S.
>>>
>>> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
>>>
>>>> Hello, Alexey.
>>>>
>>>> I just repeat our agreement to be on the same page
>>>>
>>>>> The confirmation should only present in the user-facing interfaces.
>>>> 1. We should add —force flag to the command.sh deactivation command.
>>>> 2. We should throw the exception if cluster has in-memory caches and
>>>> —force=false.
>>>> 3. We shouldn’t change Java API for deactivation.
>>>>
>>>> Is it correct?
>>>>
>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it
>>>> I think it because the command itself has a «DROP» word in it’s name.
>>>> Which clearly explains what will happen on it’s execution.
>>>>
>>>> On the opposite «deactivation» command doesn’t have any sign of
>>>> destructive operation.
>>>> That’s why we should warn user about it’s consequences.
>>>>
>>>>
>>>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
>> alexey.goncharuk@gmail.com>
>>>> написал(а):
>>>>> Igniters, Ivan, Nikolay,
>>>>>
>>>>> I am strongly against adding confirmation flags to any kind of APIs,
>>>>> whether we change the deactivation behavior or not (even though I agree
>>>>> that it makes sense to fix the deactivation to not clean up the
>> in-memory
>>>>> data). The confirmation should only present in the user-facing
>>>> interfaces.
>>>>> I cannot recall any software interface which has such a flag. None of
>> the
>>>>> syscalls which delete files (a very destructive operation) have this
>>>> flag.
>>>>> The DROP TABLE command does not have a "yes I am sure" clause in it.
>> As I
>>>>> already mentioned, when used programmatically, most users will likely
>>>>> simply pass 'true' as the new flag because they already know the
>>>> behavior.
>>>>> This is a clear sign of a bad design choice.
>>>>>
>>>>> On top of that, given that it is our intention to change the behavior
>> of
>>>>> deactivation to not loose the in-memory data, it does not make sense to
>>>> me
>>>>> to change the API.
>>>>
>>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hello Nikolay,

I am talking about the interactive mode of the control utility, which
requires explicit confirmation from the user.
Please take a look at DeactivateCommand#prepareConfirmation and its usages.
It seems to me, this mode has the same aim as the forceDeactivation flag.
We can change the message returned by DeactivateCommand#confirmationPrompt
as follows:
    "Warning: the command will deactivate the cluster nnn and clear
in-memory caches (without persistence) including system caches."

What do you think?

Thanks,
S.

вт, 24 мар. 2020 г. в 13:07, Nikolay Izhikov <ni...@apache.org>:

> Hello, Slava.
>
> Are you talking about this commit [1] (sorry for commit message it’s due
> to the Github issue)?
>
> The message for this command for now
>
> «Deactivation stopped. Deactivation clears in-memory caches (without
> persistence) including the system caches.»
>
> Is it clear enough?
>
> [1]
> https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a
>
>
> > 24 марта 2020 г., в 13:02, Вячеслав Коптилин <sl...@gmail.com>
> написал(а):
> >
> > Hi Nikolay,
> >
> >> 1. We should add —force flag to the command.sh deactivation command.
> > I just checked and it seems that the deactivation command
> > (control-utility.sh) already has a confirmation option.
> > Perhaps, we need to clearly state the consequences of using this command
> > with in-memory caches.
> >
> > Thanks,
> > S.
> >
> > вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
> >
> >> Hello, Alexey.
> >>
> >> I just repeat our agreement to be on the same page
> >>
> >>> The confirmation should only present in the user-facing interfaces.
> >>
> >> 1. We should add —force flag to the command.sh deactivation command.
> >> 2. We should throw the exception if cluster has in-memory caches and
> >> —force=false.
> >> 3. We shouldn’t change Java API for deactivation.
> >>
> >> Is it correct?
> >>
> >>> The DROP TABLE command does not have a "yes I am sure" clause in it
> >>
> >> I think it because the command itself has a «DROP» word in it’s name.
> >> Which clearly explains what will happen on it’s execution.
> >>
> >> On the opposite «deactivation» command doesn’t have any sign of
> >> destructive operation.
> >> That’s why we should warn user about it’s consequences.
> >>
> >>
> >>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> >> написал(а):
> >>>
> >>> Igniters, Ivan, Nikolay,
> >>>
> >>> I am strongly against adding confirmation flags to any kind of APIs,
> >>> whether we change the deactivation behavior or not (even though I agree
> >>> that it makes sense to fix the deactivation to not clean up the
> in-memory
> >>> data). The confirmation should only present in the user-facing
> >> interfaces.
> >>> I cannot recall any software interface which has such a flag. None of
> the
> >>> syscalls which delete files (a very destructive operation) have this
> >> flag.
> >>> The DROP TABLE command does not have a "yes I am sure" clause in it.
> As I
> >>> already mentioned, when used programmatically, most users will likely
> >>> simply pass 'true' as the new flag because they already know the
> >> behavior.
> >>> This is a clear sign of a bad design choice.
> >>>
> >>> On top of that, given that it is our intention to change the behavior
> of
> >>> deactivation to not loose the in-memory data, it does not make sense to
> >> me
> >>> to change the API.
> >>
> >>
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Slava.

Are you talking about this commit [1] (sorry for commit message it’s due to the Github issue)?

The message for this command for now

«Deactivation stopped. Deactivation clears in-memory caches (without persistence) including the system caches.» 

Is it clear enough?

[1] https://github.com/apache/ignite/commit/4921fcf1fecbd8a1ab02099e09cc2adb0b3ff88a


> 24 марта 2020 г., в 13:02, Вячеслав Коптилин <sl...@gmail.com> написал(а):
> 
> Hi Nikolay,
> 
>> 1. We should add —force flag to the command.sh deactivation command.
> I just checked and it seems that the deactivation command
> (control-utility.sh) already has a confirmation option.
> Perhaps, we need to clearly state the consequences of using this command
> with in-memory caches.
> 
> Thanks,
> S.
> 
> вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:
> 
>> Hello, Alexey.
>> 
>> I just repeat our agreement to be on the same page
>> 
>>> The confirmation should only present in the user-facing interfaces.
>> 
>> 1. We should add —force flag to the command.sh deactivation command.
>> 2. We should throw the exception if cluster has in-memory caches and
>> —force=false.
>> 3. We shouldn’t change Java API for deactivation.
>> 
>> Is it correct?
>> 
>>> The DROP TABLE command does not have a "yes I am sure" clause in it
>> 
>> I think it because the command itself has a «DROP» word in it’s name.
>> Which clearly explains what will happen on it’s execution.
>> 
>> On the opposite «deactivation» command doesn’t have any sign of
>> destructive operation.
>> That’s why we should warn user about it’s consequences.
>> 
>> 
>>> 24 марта 2020 г., в 12:38, Alexey Goncharuk <al...@gmail.com>
>> написал(а):
>>> 
>>> Igniters, Ivan, Nikolay,
>>> 
>>> I am strongly against adding confirmation flags to any kind of APIs,
>>> whether we change the deactivation behavior or not (even though I agree
>>> that it makes sense to fix the deactivation to not clean up the in-memory
>>> data). The confirmation should only present in the user-facing
>> interfaces.
>>> I cannot recall any software interface which has such a flag. None of the
>>> syscalls which delete files (a very destructive operation) have this
>> flag.
>>> The DROP TABLE command does not have a "yes I am sure" clause in it. As I
>>> already mentioned, when used programmatically, most users will likely
>>> simply pass 'true' as the new flag because they already know the
>> behavior.
>>> This is a clear sign of a bad design choice.
>>> 
>>> On top of that, given that it is our intention to change the behavior of
>>> deactivation to not loose the in-memory data, it does not make sense to
>> me
>>> to change the API.
>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hi Nikolay,

> 1. We should add —force flag to the command.sh deactivation command.
I just checked and it seems that the deactivation command
(control-utility.sh) already has a confirmation option.
Perhaps, we need to clearly state the consequences of using this command
with in-memory caches.

Thanks,
S.

вт, 24 мар. 2020 г. в 12:51, Nikolay Izhikov <ni...@apache.org>:

> Hello, Alexey.
>
> I just repeat our agreement to be on the same page
>
> > The confirmation should only present in the user-facing interfaces.
>
> 1. We should add —force flag to the command.sh deactivation command.
> 2. We should throw the exception if cluster has in-memory caches and
> —force=false.
> 3. We shouldn’t change Java API for deactivation.
>
> Is it correct?
>
> > The DROP TABLE command does not have a "yes I am sure" clause in it
>
> I think it because the command itself has a «DROP» word in it’s name.
> Which clearly explains what will happen on it’s execution.
>
> On the opposite «deactivation» command doesn’t have any sign of
> destructive operation.
> That’s why we should warn user about it’s consequences.
>
>
> > 24 марта 2020 г., в 12:38, Alexey Goncharuk <al...@gmail.com>
> написал(а):
> >
> > Igniters, Ivan, Nikolay,
> >
> > I am strongly against adding confirmation flags to any kind of APIs,
> > whether we change the deactivation behavior or not (even though I agree
> > that it makes sense to fix the deactivation to not clean up the in-memory
> > data). The confirmation should only present in the user-facing
> interfaces.
> > I cannot recall any software interface which has such a flag. None of the
> > syscalls which delete files (a very destructive operation) have this
> flag.
> > The DROP TABLE command does not have a "yes I am sure" clause in it. As I
> > already mentioned, when used programmatically, most users will likely
> > simply pass 'true' as the new flag because they already know the
> behavior.
> > This is a clear sign of a bad design choice.
> >
> > On top of that, given that it is our intention to change the behavior of
> > deactivation to not loose the in-memory data, it does not make sense to
> me
> > to change the API.
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

>
> Hello, Alexey.
>
> I just repeat our agreement to be on the same page
>
> > The confirmation should only present in the user-facing interfaces.
>
> 1. We should add —force flag to the command.sh deactivation command.
> 2. We should throw the exception if cluster has in-memory caches and
> —force=false.
> 3. We shouldn’t change Java API for deactivation.
>
> Is it correct?
>

Yes, this is what I had in mind.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Alexey.

I just repeat our agreement to be on the same page

> The confirmation should only present in the user-facing interfaces.

1. We should add —force flag to the command.sh deactivation command.
2. We should throw the exception if cluster has in-memory caches and —force=false.
3. We shouldn’t change Java API for deactivation.

Is it correct?

> The DROP TABLE command does not have a "yes I am sure" clause in it

I think it because the command itself has a «DROP» word in it’s name.
Which clearly explains what will happen on it’s execution.

On the opposite «deactivation» command doesn’t have any sign of destructive operation.
That’s why we should warn user about it’s consequences.


> 24 марта 2020 г., в 12:38, Alexey Goncharuk <al...@gmail.com> написал(а):
> 
> Igniters, Ivan, Nikolay,
> 
> I am strongly against adding confirmation flags to any kind of APIs,
> whether we change the deactivation behavior or not (even though I agree
> that it makes sense to fix the deactivation to not clean up the in-memory
> data). The confirmation should only present in the user-facing interfaces.
> I cannot recall any software interface which has such a flag. None of the
> syscalls which delete files (a very destructive operation) have this flag.
> The DROP TABLE command does not have a "yes I am sure" clause in it. As I
> already mentioned, when used programmatically, most users will likely
> simply pass 'true' as the new flag because they already know the behavior.
> This is a clear sign of a bad design choice.
> 
> On top of that, given that it is our intention to change the behavior of
> deactivation to not loose the in-memory data, it does not make sense to me
> to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

Igniters, Ivan, Nikolay,

I am strongly against adding confirmation flags to any kind of APIs,
whether we change the deactivation behavior or not (even though I agree
that it makes sense to fix the deactivation to not clean up the in-memory
data). The confirmation should only present in the user-facing interfaces.
I cannot recall any software interface which has such a flag. None of the
syscalls which delete files (a very destructive operation) have this flag.
The DROP TABLE command does not have a "yes I am sure" clause in it. As I
already mentioned, when used programmatically, most users will likely
simply pass 'true' as the new flag because they already know the behavior.
This is a clear sign of a bad design choice.

On top of that, given that it is our intention to change the behavior of
deactivation to not loose the in-memory data, it does not make sense to me
to change the API.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

>
> I can’t agree with the «temporary» design.
> We have neither design nor IEP or contributor who can fix current behavior.
> And, if I understand Alexey Goncharyuk correctly, current behavior was
> implemented intentionally.

 Alex, what do you think? Are we on the same page that desired behavior for
the deactivation is to keep data of all in-memory caches, even though it
was intentionally implemented in 2.0 another way?

On Tue, Mar 24, 2020 at 12:21 PM Nikolay Izhikov <ni...@apache.org>
wrote:

> Hello, Ivan.
>
> > I believe we should fix the issue instead of adapting API to temporary
> flaws.
>
> Agree. Let’s fix it.
>
> >  I think that clear description of active(false) impact in the
> documentation is more than enough
>
> I can’t agree with this point.
>
> We shouldn’t imply the assumption that every user reads the whole
> documentation and completely understand the consequences of the
> deactivation command.
>
> This whole thread shows that even active core developers don't understand
> it.
>
> So my proposal is to remove --force flag only after we fix deactivation.
>
> > To sum it up, the question is whether we should reflect temporary system
> design flaws in the API
>
> I can’t agree with the «temporary» design.
> We have neither design nor IEP or contributor who can fix current behavior.
> And, if I understand Alexey Goncharyuk correctly, current behavior was
> implemented intentionally.
>
> So, my understanding that current implementation would be here for a while.
> And after we fix it I totally support removing —force flag.
>
> > 24 марта 2020 г., в 12:06, Ivan Rakov <iv...@gmail.com>
> написал(а):
> >
> >>
> >> I think the only question is - Do we need —force flag in Java API or
> not.
> >
> > From my perspective, there's also no agreement that it should be present
> > in the thin clients' API. For instance, I think it shouldn't.
> >
> > As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other*
> purpose.
> >> Can you provide a simple reproducer when in-memory data not cleared on
> >> deactivation?
> >
> > Preserving in-memory data isn't implemented so far, so I can't provide a
> > reproducer. My point is that we are halfway through it: we can build a
> > solution based on IGNITE_REUSE_MEMORY_ON_DEACTIVATE and additional logic
> > with reusing memory pages.
> >
> > For me, the ultimate value of Ignite into real production environment is
> >> user data.
> >> If we have some cases when data is lost - we should avoid it as hard as
> we
> >> can.
> >>
> >> So, for now, this flag required.
> >
> > Totally agree that sudden vanishing of user data is unacceptable. But I
> > don't see how it implies that we have to solve this issue by tangling
> > public API. If we see that system behaves incorrectly, I believe we
> should
> > fix the issue instead of adapting API to temporary flaws. I think that
> > clear description of active(false) impact in the documentation is more
> than
> > enough: on the one hand, if user didn't read documentation for the method
> > he calls, he can't complain about the consequences; on the other hand, if
> > user decided to deactivate the cluster for no matter what, -force flag
> will
> > barely stop him.
> > We anyway have enough time before 2.9 to implement a proper solution.
> >
> > To sum it up, the question is whether we should reflect temporary system
> > design flaws in the API. I think, we surely shouldn't: API certainly
> lives
> > longer and is not intended to collect workarounds for all bugs that are
> > already fixed or planned to be fixed.
> > We can collect more opinions on this.
> >
> > On Tue, Mar 24, 2020 at 10:22 AM Nikolay Izhikov <ni...@apache.org>
> > wrote:
> >
> >> Alexey.
> >>
> >> Having the way to silently vanish user data is even worse.
> >> So I’m strictly against removing —force flag.
> >>
> >>> 24 марта 2020 г., в 10:16, Alexei Scherbakov <
> >> alexey.scherbakoff@gmail.com> написал(а):
> >>>
> >>> Nikolay,
> >>>
> >>> I'm on the same page with Ivan.
> >>>
> >>> Having "force" flag in public API as preposterous as having it in
> >>> System.exit.
> >>> For me it looks like badly designed API.
> >>> If a call to some method is dangerous it should be clearly specified in
> >> the
> >>> javadoc.
> >>> I'm also against some "temporary" API.
> >>>
> >>> We should:
> >>>
> >>> 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh
> >> for a
> >>> long time has support for a confirmation on deactivation (interactive
> >> mode).
> >>> 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory
> >> content
> >>> after deactivation. We should start working on restoring page memory
> >> state
> >>> after subsequent reactivation.
> >>> 3. Describe current behavior for in-memory cache on deactivation in
> >> Ignite
> >>> documentation.
> >>>
> >>>
> >>> пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <ni...@apache.org>:
> >>>
> >>>> Hello, Ivan.
> >>>>
> >>>>> Seems like we don't have a final agreement on whether we should add
> >> force
> >>>> flag to deactivation API.
> >>>>
> >>>> I think the only question is - Do we need —force flag in Java API or
> >> not.
> >>>>
> >>>>
> >>>>> As a final solution, I'd want to see behavior when all in-memory data
> >> is
> >>>> available after deactivation and further activation.
> >>>>
> >>>> Agree.
> >>>>
> >>>>> I believe it’s possible to don't deallocate memory
> >>>>> (like mentioned before, we already can achieve that with
> >>>> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded
> >> data
> >>>> pages on next activation and caches start.
> >>>>
> >>>> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other*
> >> purpose.
> >>>> Can you provide a simple reproducer when in-memory data not cleared on
> >>>> deactivation?
> >>>>
> >>>>> Considering this, do we really need to introduce force flag as a
> >>>> temporary precaution?
> >>>>
> >>>> My answer is yes we need it.
> >>>> Right now, we can’t prevent data loss on deactivation for in-memory
> >> caches.
> >>>>
> >>>> For me, the ultimate value of Ignite into real production environment
> is
> >>>> user data.
> >>>> If we have some cases when data is lost - we should avoid it as hard
> as
> >> we
> >>>> can.
> >>>>
> >>>> So, for now, this flag required.
> >>>>
> >>>>> I suggest to rollback [2] from AI master, stop working on [1] and
> focus
> >>>> on how to implement keeping in-memory data after the deactivation.
> >>>>
> >>>> I think we can remove —force flag only after implementation of keeping
> >>>> in-memory data on deactivation would be finished.
> >>>> If that happens before 2.9 then we can have clearer API.
> >>>>
> >>>>> 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com>
> >>>> написал(а):
> >>>>>
> >>>>> Folks,
> >>>>>
> >>>>> Let's revive this discussion until it's too late and all API changes
> >> are
> >>>>> merged to master [1].
> >>>>> Seems like we don't have a final agreement on whether we should add
> >> force
> >>>>> flag to deactivation API.
> >>>>>
> >>>>> First of all, I think we are all on the same page that in-memory
> caches
> >>>>> data vanishing on deactivation is counter-intuitive and dangerous.
> As a
> >>>>> final solution, I'd want to see behavior when all in-memory data is
> >>>>> available after deactivation and further activation. I believe it's
> >>>>> possible to don't deallocate memory (like mentioned before, we
> already
> >>>> can
> >>>>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and
> carefully
> >>>>> reuse all loaded data pages on next activation and caches start.
> >>>>>
> >>>>> Also, this is a wider question, but: do we understand what cluster
> >>>>> deactivation is actually intended for? I can only think of two cases:
> >>>>> - graceful cluster shutdown: an ability to cut checkpoints and to end
> >>>>> transactional load consistently prior to further stop of all nodes
> >>>>> - blocking all API (both reads and writes) due to some maintenance
> >>>>> Neither of them requires forcefully clearing all in-memory data on
> >>>>> deactivation. If everyone agrees, from now on we should assume data
> >>>>> clearing as system design flaw that should be fixed, not as possible
> >>>>> scenario which we should support on the API level.
> >>>>>
> >>>>> Considering this, do we really need to introduce force flag as a
> >>>> temporary
> >>>>> precaution? I have at least two reasons against it:
> >>>>> 1) Once API was changed and released, we have to support it until the
> >>>> next
> >>>>> major release. If we all understand that data vanishing issue is
> >>>> fixable, I
> >>>>> believe we shouldn't engrave in the API flags that will become
> >> pointless.
> >>>>> 2) More personal one, but I'm against any force flags in the API.
> This
> >>>>> makes API harder to understand; more than that, the presence of such
> >>>> flags
> >>>>> just highlights that API is poorly designed.
> >>>>>
> >>>>> I suggest to rollback [2] from AI master, stop working on [1] and
> focus
> >>>> on
> >>>>> how to implement keeping in-memory data after the deactivation.
> >>>>> I think we can still require user consent for deactivation via
> >> control.sh
> >>>>> (it already requires --yes) and JMX.
> >>>>>
> >>>>> Thoughts?
> >>>>>
> >>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614
> >>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701
> >>>>>
> >>>>> --
> >>>>> Ivan
> >>>>>
> >>>>>
> >>>>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vladsz83@gmail.com
> >
> >>>> wrote:
> >>>>>
> >>>>>> Nikolay, I think we should reconsider clearing at least system
> caches
> >>>>>> when deactivating.
> >>>>>>
> >>>>>> 17.03.2020 14:18, Nikolay Izhikov пишет:
> >>>>>>> Hello, Vladimir.
> >>>>>>>
> >>>>>>> I don’t get it.
> >>>>>>>
> >>>>>>> What is your proposal?
> >>>>>>> What we should do?
> >>>>>>>
> >>>>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
> >>>>>> написал(а):
> >>>>>>>>
> >>>>>>>> Nikolay, hi.
> >>>>>>>>
> >>>>>>>>>>> And should be covered with the  —force parameter we added.
> >>>>>>>> As fix for user cases - yes. My idea is to emphasize overall
> ability
> >>>> to
> >>>>>> lose various objects, not only data. Probably might be reconsidered
> in
> >>>>>> future.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
> >>>>>>>>> Hello, Vladimir.
> >>>>>>>>>
> >>>>>>>>> If there is at lease one persistent data region then system data
> >>>>>> region also becomes persistent.
> >>>>>>>>> Your example applies only to pure in-memory clusters.
> >>>>>>>>>
> >>>>>>>>> And should be covered with the —force parameter we added.
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vladsz83@gmail.com
> >
> >>>>>> написал(а):
> >>>>>>>>>>
> >>>>>>>>>>   Hi, all.
> >>>>>>>>>>
> >>>>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone
> >>>> take
> >>>>>> a look to the previous email with an issue? Isn't this conductvery
> >>>> wierd?
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>>
> >>> Best regards,
> >>> Alexei Scherbakov
> >>
> >>
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Ivan.

> I believe we should fix the issue instead of adapting API to temporary flaws.

Agree. Let’s fix it.

>  I think that clear description of active(false) impact in the documentation is more than enough

I can’t agree with this point.

We shouldn’t imply the assumption that every user reads the whole documentation and completely understand the consequences of the 
deactivation command.

This whole thread shows that even active core developers don't understand it.

So my proposal is to remove --force flag only after we fix deactivation.

> To sum it up, the question is whether we should reflect temporary system design flaws in the API

I can’t agree with the «temporary» design.
We have neither design nor IEP or contributor who can fix current behavior.
And, if I understand Alexey Goncharyuk correctly, current behavior was implemented intentionally.

So, my understanding that current implementation would be here for a while.
And after we fix it I totally support removing —force flag.

> 24 марта 2020 г., в 12:06, Ivan Rakov <iv...@gmail.com> написал(а):
> 
>> 
>> I think the only question is - Do we need —force flag in Java API or not.
> 
> From my perspective, there's also no agreement that it should be present
> in the thin clients' API. For instance, I think it shouldn't.
> 
> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
>> Can you provide a simple reproducer when in-memory data not cleared on
>> deactivation?
> 
> Preserving in-memory data isn't implemented so far, so I can't provide a
> reproducer. My point is that we are halfway through it: we can build a
> solution based on IGNITE_REUSE_MEMORY_ON_DEACTIVATE and additional logic
> with reusing memory pages.
> 
> For me, the ultimate value of Ignite into real production environment is
>> user data.
>> If we have some cases when data is lost - we should avoid it as hard as we
>> can.
>> 
>> So, for now, this flag required.
> 
> Totally agree that sudden vanishing of user data is unacceptable. But I
> don't see how it implies that we have to solve this issue by tangling
> public API. If we see that system behaves incorrectly, I believe we should
> fix the issue instead of adapting API to temporary flaws. I think that
> clear description of active(false) impact in the documentation is more than
> enough: on the one hand, if user didn't read documentation for the method
> he calls, he can't complain about the consequences; on the other hand, if
> user decided to deactivate the cluster for no matter what, -force flag will
> barely stop him.
> We anyway have enough time before 2.9 to implement a proper solution.
> 
> To sum it up, the question is whether we should reflect temporary system
> design flaws in the API. I think, we surely shouldn't: API certainly lives
> longer and is not intended to collect workarounds for all bugs that are
> already fixed or planned to be fixed.
> We can collect more opinions on this.
> 
> On Tue, Mar 24, 2020 at 10:22 AM Nikolay Izhikov <ni...@apache.org>
> wrote:
> 
>> Alexey.
>> 
>> Having the way to silently vanish user data is even worse.
>> So I’m strictly against removing —force flag.
>> 
>>> 24 марта 2020 г., в 10:16, Alexei Scherbakov <
>> alexey.scherbakoff@gmail.com> написал(а):
>>> 
>>> Nikolay,
>>> 
>>> I'm on the same page with Ivan.
>>> 
>>> Having "force" flag in public API as preposterous as having it in
>>> System.exit.
>>> For me it looks like badly designed API.
>>> If a call to some method is dangerous it should be clearly specified in
>> the
>>> javadoc.
>>> I'm also against some "temporary" API.
>>> 
>>> We should:
>>> 
>>> 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh
>> for a
>>> long time has support for a confirmation on deactivation (interactive
>> mode).
>>> 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory
>> content
>>> after deactivation. We should start working on restoring page memory
>> state
>>> after subsequent reactivation.
>>> 3. Describe current behavior for in-memory cache on deactivation in
>> Ignite
>>> documentation.
>>> 
>>> 
>>> пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <ni...@apache.org>:
>>> 
>>>> Hello, Ivan.
>>>> 
>>>>> Seems like we don't have a final agreement on whether we should add
>> force
>>>> flag to deactivation API.
>>>> 
>>>> I think the only question is - Do we need —force flag in Java API or
>> not.
>>>> 
>>>> 
>>>>> As a final solution, I'd want to see behavior when all in-memory data
>> is
>>>> available after deactivation and further activation.
>>>> 
>>>> Agree.
>>>> 
>>>>> I believe it’s possible to don't deallocate memory
>>>>> (like mentioned before, we already can achieve that with
>>>> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded
>> data
>>>> pages on next activation and caches start.
>>>> 
>>>> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other*
>> purpose.
>>>> Can you provide a simple reproducer when in-memory data not cleared on
>>>> deactivation?
>>>> 
>>>>> Considering this, do we really need to introduce force flag as a
>>>> temporary precaution?
>>>> 
>>>> My answer is yes we need it.
>>>> Right now, we can’t prevent data loss on deactivation for in-memory
>> caches.
>>>> 
>>>> For me, the ultimate value of Ignite into real production environment is
>>>> user data.
>>>> If we have some cases when data is lost - we should avoid it as hard as
>> we
>>>> can.
>>>> 
>>>> So, for now, this flag required.
>>>> 
>>>>> I suggest to rollback [2] from AI master, stop working on [1] and focus
>>>> on how to implement keeping in-memory data after the deactivation.
>>>> 
>>>> I think we can remove —force flag only after implementation of keeping
>>>> in-memory data on deactivation would be finished.
>>>> If that happens before 2.9 then we can have clearer API.
>>>> 
>>>>> 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com>
>>>> написал(а):
>>>>> 
>>>>> Folks,
>>>>> 
>>>>> Let's revive this discussion until it's too late and all API changes
>> are
>>>>> merged to master [1].
>>>>> Seems like we don't have a final agreement on whether we should add
>> force
>>>>> flag to deactivation API.
>>>>> 
>>>>> First of all, I think we are all on the same page that in-memory caches
>>>>> data vanishing on deactivation is counter-intuitive and dangerous. As a
>>>>> final solution, I'd want to see behavior when all in-memory data is
>>>>> available after deactivation and further activation. I believe it's
>>>>> possible to don't deallocate memory (like mentioned before, we already
>>>> can
>>>>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
>>>>> reuse all loaded data pages on next activation and caches start.
>>>>> 
>>>>> Also, this is a wider question, but: do we understand what cluster
>>>>> deactivation is actually intended for? I can only think of two cases:
>>>>> - graceful cluster shutdown: an ability to cut checkpoints and to end
>>>>> transactional load consistently prior to further stop of all nodes
>>>>> - blocking all API (both reads and writes) due to some maintenance
>>>>> Neither of them requires forcefully clearing all in-memory data on
>>>>> deactivation. If everyone agrees, from now on we should assume data
>>>>> clearing as system design flaw that should be fixed, not as possible
>>>>> scenario which we should support on the API level.
>>>>> 
>>>>> Considering this, do we really need to introduce force flag as a
>>>> temporary
>>>>> precaution? I have at least two reasons against it:
>>>>> 1) Once API was changed and released, we have to support it until the
>>>> next
>>>>> major release. If we all understand that data vanishing issue is
>>>> fixable, I
>>>>> believe we shouldn't engrave in the API flags that will become
>> pointless.
>>>>> 2) More personal one, but I'm against any force flags in the API. This
>>>>> makes API harder to understand; more than that, the presence of such
>>>> flags
>>>>> just highlights that API is poorly designed.
>>>>> 
>>>>> I suggest to rollback [2] from AI master, stop working on [1] and focus
>>>> on
>>>>> how to implement keeping in-memory data after the deactivation.
>>>>> I think we can still require user consent for deactivation via
>> control.sh
>>>>> (it already requires --yes) and JMX.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614
>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701
>>>>> 
>>>>> --
>>>>> Ivan
>>>>> 
>>>>> 
>>>>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> Nikolay, I think we should reconsider clearing at least system caches
>>>>>> when deactivating.
>>>>>> 
>>>>>> 17.03.2020 14:18, Nikolay Izhikov пишет:
>>>>>>> Hello, Vladimir.
>>>>>>> 
>>>>>>> I don’t get it.
>>>>>>> 
>>>>>>> What is your proposal?
>>>>>>> What we should do?
>>>>>>> 
>>>>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
>>>>>> написал(а):
>>>>>>>> 
>>>>>>>> Nikolay, hi.
>>>>>>>> 
>>>>>>>>>>> And should be covered with the  —force parameter we added.
>>>>>>>> As fix for user cases - yes. My idea is to emphasize overall ability
>>>> to
>>>>>> lose various objects, not only data. Probably might be reconsidered in
>>>>>> future.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
>>>>>>>>> Hello, Vladimir.
>>>>>>>>> 
>>>>>>>>> If there is at lease one persistent data region then system data
>>>>>> region also becomes persistent.
>>>>>>>>> Your example applies only to pure in-memory clusters.
>>>>>>>>> 
>>>>>>>>> And should be covered with the —force parameter we added.
>>>>>>>>> 
>>>>>>>>> What do you think?
>>>>>>>>> 
>>>>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
>>>>>> написал(а):
>>>>>>>>>> 
>>>>>>>>>>   Hi, all.
>>>>>>>>>> 
>>>>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone
>>>> take
>>>>>> a look to the previous email with an issue? Isn't this conductvery
>>>> wierd?
>>>>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>>> --
>>> 
>>> Best regards,
>>> Alexei Scherbakov
>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

>
> I think the only question is - Do we need —force flag in Java API or not.

 From my perspective, there's also no agreement that it should be present
in the thin clients' API. For instance, I think it shouldn't.

As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
> Can you provide a simple reproducer when in-memory data not cleared on
> deactivation?

 Preserving in-memory data isn't implemented so far, so I can't provide a
reproducer. My point is that we are halfway through it: we can build a
solution based on IGNITE_REUSE_MEMORY_ON_DEACTIVATE and additional logic
with reusing memory pages.

For me, the ultimate value of Ignite into real production environment is
> user data.
> If we have some cases when data is lost - we should avoid it as hard as we
> can.
>
> So, for now, this flag required.

Totally agree that sudden vanishing of user data is unacceptable. But I
don't see how it implies that we have to solve this issue by tangling
public API. If we see that system behaves incorrectly, I believe we should
fix the issue instead of adapting API to temporary flaws. I think that
clear description of active(false) impact in the documentation is more than
enough: on the one hand, if user didn't read documentation for the method
he calls, he can't complain about the consequences; on the other hand, if
user decided to deactivate the cluster for no matter what, -force flag will
barely stop him.
We anyway have enough time before 2.9 to implement a proper solution.

To sum it up, the question is whether we should reflect temporary system
design flaws in the API. I think, we surely shouldn't: API certainly lives
longer and is not intended to collect workarounds for all bugs that are
already fixed or planned to be fixed.
We can collect more opinions on this.

On Tue, Mar 24, 2020 at 10:22 AM Nikolay Izhikov <ni...@apache.org>
wrote:

> Alexey.
>
> Having the way to silently vanish user data is even worse.
> So I’m strictly against removing —force flag.
>
> > 24 марта 2020 г., в 10:16, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> написал(а):
> >
> > Nikolay,
> >
> > I'm on the same page with Ivan.
> >
> > Having "force" flag in public API as preposterous as having it in
> > System.exit.
> > For me it looks like badly designed API.
> > If a call to some method is dangerous it should be clearly specified in
> the
> > javadoc.
> > I'm also against some "temporary" API.
> >
> > We should:
> >
> > 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh
> for a
> > long time has support for a confirmation on deactivation (interactive
> mode).
> > 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory
> content
> > after deactivation. We should start working on restoring page memory
> state
> > after subsequent reactivation.
> > 3. Describe current behavior for in-memory cache on deactivation in
> Ignite
> > documentation.
> >
> >
> > пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <ni...@apache.org>:
> >
> >> Hello, Ivan.
> >>
> >>> Seems like we don't have a final agreement on whether we should add
> force
> >> flag to deactivation API.
> >>
> >> I think the only question is - Do we need —force flag in Java API or
> not.
> >>
> >>
> >>> As a final solution, I'd want to see behavior when all in-memory data
> is
> >> available after deactivation and further activation.
> >>
> >> Agree.
> >>
> >>> I believe it’s possible to don't deallocate memory
> >>> (like mentioned before, we already can achieve that with
> >> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded
> data
> >> pages on next activation and caches start.
> >>
> >> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other*
> purpose.
> >> Can you provide a simple reproducer when in-memory data not cleared on
> >> deactivation?
> >>
> >>> Considering this, do we really need to introduce force flag as a
> >> temporary precaution?
> >>
> >> My answer is yes we need it.
> >> Right now, we can’t prevent data loss on deactivation for in-memory
> caches.
> >>
> >> For me, the ultimate value of Ignite into real production environment is
> >> user data.
> >> If we have some cases when data is lost - we should avoid it as hard as
> we
> >> can.
> >>
> >> So, for now, this flag required.
> >>
> >>> I suggest to rollback [2] from AI master, stop working on [1] and focus
> >> on how to implement keeping in-memory data after the deactivation.
> >>
> >> I think we can remove —force flag only after implementation of keeping
> >> in-memory data on deactivation would be finished.
> >> If that happens before 2.9 then we can have clearer API.
> >>
> >>> 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com>
> >> написал(а):
> >>>
> >>> Folks,
> >>>
> >>> Let's revive this discussion until it's too late and all API changes
> are
> >>> merged to master [1].
> >>> Seems like we don't have a final agreement on whether we should add
> force
> >>> flag to deactivation API.
> >>>
> >>> First of all, I think we are all on the same page that in-memory caches
> >>> data vanishing on deactivation is counter-intuitive and dangerous. As a
> >>> final solution, I'd want to see behavior when all in-memory data is
> >>> available after deactivation and further activation. I believe it's
> >>> possible to don't deallocate memory (like mentioned before, we already
> >> can
> >>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
> >>> reuse all loaded data pages on next activation and caches start.
> >>>
> >>> Also, this is a wider question, but: do we understand what cluster
> >>> deactivation is actually intended for? I can only think of two cases:
> >>> - graceful cluster shutdown: an ability to cut checkpoints and to end
> >>> transactional load consistently prior to further stop of all nodes
> >>> - blocking all API (both reads and writes) due to some maintenance
> >>> Neither of them requires forcefully clearing all in-memory data on
> >>> deactivation. If everyone agrees, from now on we should assume data
> >>> clearing as system design flaw that should be fixed, not as possible
> >>> scenario which we should support on the API level.
> >>>
> >>> Considering this, do we really need to introduce force flag as a
> >> temporary
> >>> precaution? I have at least two reasons against it:
> >>> 1) Once API was changed and released, we have to support it until the
> >> next
> >>> major release. If we all understand that data vanishing issue is
> >> fixable, I
> >>> believe we shouldn't engrave in the API flags that will become
> pointless.
> >>> 2) More personal one, but I'm against any force flags in the API. This
> >>> makes API harder to understand; more than that, the presence of such
> >> flags
> >>> just highlights that API is poorly designed.
> >>>
> >>> I suggest to rollback [2] from AI master, stop working on [1] and focus
> >> on
> >>> how to implement keeping in-memory data after the deactivation.
> >>> I think we can still require user consent for deactivation via
> control.sh
> >>> (it already requires --yes) and JMX.
> >>>
> >>> Thoughts?
> >>>
> >>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614
> >>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701
> >>>
> >>> --
> >>> Ivan
> >>>
> >>>
> >>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com>
> >> wrote:
> >>>
> >>>> Nikolay, I think we should reconsider clearing at least system caches
> >>>> when deactivating.
> >>>>
> >>>> 17.03.2020 14:18, Nikolay Izhikov пишет:
> >>>>> Hello, Vladimir.
> >>>>>
> >>>>> I don’t get it.
> >>>>>
> >>>>> What is your proposal?
> >>>>> What we should do?
> >>>>>
> >>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
> >>>> написал(а):
> >>>>>>
> >>>>>> Nikolay, hi.
> >>>>>>
> >>>>>>>>> And should be covered with the  —force parameter we added.
> >>>>>> As fix for user cases - yes. My idea is to emphasize overall ability
> >> to
> >>>> lose various objects, not only data. Probably might be reconsidered in
> >>>> future.
> >>>>>>
> >>>>>>
> >>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
> >>>>>>> Hello, Vladimir.
> >>>>>>>
> >>>>>>> If there is at lease one persistent data region then system data
> >>>> region also becomes persistent.
> >>>>>>> Your example applies only to pure in-memory clusters.
> >>>>>>>
> >>>>>>> And should be covered with the —force parameter we added.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
> >>>> написал(а):
> >>>>>>>>
> >>>>>>>>    Hi, all.
> >>>>>>>>
> >>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone
> >> take
> >>>> a look to the previous email with an issue? Isn't this conductvery
> >> wierd?
> >>>>>>>>
> >>>>
> >>
> >>
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Alexey.

Having the way to silently vanish user data is even worse.
So I’m strictly against removing —force flag.

> 24 марта 2020 г., в 10:16, Alexei Scherbakov <al...@gmail.com> написал(а):
> 
> Nikolay,
> 
> I'm on the same page with Ivan.
> 
> Having "force" flag in public API as preposterous as having it in
> System.exit.
> For me it looks like badly designed API.
> If a call to some method is dangerous it should be clearly specified in the
> javadoc.
> I'm also against some "temporary" API.
> 
> We should:
> 
> 1. Partially remove IGNITE-12701 except javadoc part. Note control.sh for a
> long time has support for a confirmation on deactivation (interactive mode).
> 2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory content
> after deactivation. We should start working on restoring page memory state
> after subsequent reactivation.
> 3. Describe current behavior for in-memory cache on deactivation in Ignite
> documentation.
> 
> 
> пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <ni...@apache.org>:
> 
>> Hello, Ivan.
>> 
>>> Seems like we don't have a final agreement on whether we should add force
>> flag to deactivation API.
>> 
>> I think the only question is - Do we need —force flag in Java API or not.
>> 
>> 
>>> As a final solution, I'd want to see behavior when all in-memory data is
>> available after deactivation and further activation.
>> 
>> Agree.
>> 
>>> I believe it’s possible to don't deallocate memory
>>> (like mentioned before, we already can achieve that with
>> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded data
>> pages on next activation and caches start.
>> 
>> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
>> Can you provide a simple reproducer when in-memory data not cleared on
>> deactivation?
>> 
>>> Considering this, do we really need to introduce force flag as a
>> temporary precaution?
>> 
>> My answer is yes we need it.
>> Right now, we can’t prevent data loss on deactivation for in-memory caches.
>> 
>> For me, the ultimate value of Ignite into real production environment is
>> user data.
>> If we have some cases when data is lost - we should avoid it as hard as we
>> can.
>> 
>> So, for now, this flag required.
>> 
>>> I suggest to rollback [2] from AI master, stop working on [1] and focus
>> on how to implement keeping in-memory data after the deactivation.
>> 
>> I think we can remove —force flag only after implementation of keeping
>> in-memory data on deactivation would be finished.
>> If that happens before 2.9 then we can have clearer API.
>> 
>>> 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com>
>> написал(а):
>>> 
>>> Folks,
>>> 
>>> Let's revive this discussion until it's too late and all API changes are
>>> merged to master [1].
>>> Seems like we don't have a final agreement on whether we should add force
>>> flag to deactivation API.
>>> 
>>> First of all, I think we are all on the same page that in-memory caches
>>> data vanishing on deactivation is counter-intuitive and dangerous. As a
>>> final solution, I'd want to see behavior when all in-memory data is
>>> available after deactivation and further activation. I believe it's
>>> possible to don't deallocate memory (like mentioned before, we already
>> can
>>> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
>>> reuse all loaded data pages on next activation and caches start.
>>> 
>>> Also, this is a wider question, but: do we understand what cluster
>>> deactivation is actually intended for? I can only think of two cases:
>>> - graceful cluster shutdown: an ability to cut checkpoints and to end
>>> transactional load consistently prior to further stop of all nodes
>>> - blocking all API (both reads and writes) due to some maintenance
>>> Neither of them requires forcefully clearing all in-memory data on
>>> deactivation. If everyone agrees, from now on we should assume data
>>> clearing as system design flaw that should be fixed, not as possible
>>> scenario which we should support on the API level.
>>> 
>>> Considering this, do we really need to introduce force flag as a
>> temporary
>>> precaution? I have at least two reasons against it:
>>> 1) Once API was changed and released, we have to support it until the
>> next
>>> major release. If we all understand that data vanishing issue is
>> fixable, I
>>> believe we shouldn't engrave in the API flags that will become pointless.
>>> 2) More personal one, but I'm against any force flags in the API. This
>>> makes API harder to understand; more than that, the presence of such
>> flags
>>> just highlights that API is poorly designed.
>>> 
>>> I suggest to rollback [2] from AI master, stop working on [1] and focus
>> on
>>> how to implement keeping in-memory data after the deactivation.
>>> I think we can still require user consent for deactivation via control.sh
>>> (it already requires --yes) and JMX.
>>> 
>>> Thoughts?
>>> 
>>> [1]: https://issues.apache.org/jira/browse/IGNITE-12614
>>> [2]: https://issues.apache.org/jira/browse/IGNITE-12701
>>> 
>>> --
>>> Ivan
>>> 
>>> 
>>> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com>
>> wrote:
>>> 
>>>> Nikolay, I think we should reconsider clearing at least system caches
>>>> when deactivating.
>>>> 
>>>> 17.03.2020 14:18, Nikolay Izhikov пишет:
>>>>> Hello, Vladimir.
>>>>> 
>>>>> I don’t get it.
>>>>> 
>>>>> What is your proposal?
>>>>> What we should do?
>>>>> 
>>>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
>>>> написал(а):
>>>>>> 
>>>>>> Nikolay, hi.
>>>>>> 
>>>>>>>>> And should be covered with the  —force parameter we added.
>>>>>> As fix for user cases - yes. My idea is to emphasize overall ability
>> to
>>>> lose various objects, not only data. Probably might be reconsidered in
>>>> future.
>>>>>> 
>>>>>> 
>>>>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
>>>>>>> Hello, Vladimir.
>>>>>>> 
>>>>>>> If there is at lease one persistent data region then system data
>>>> region also becomes persistent.
>>>>>>> Your example applies only to pure in-memory clusters.
>>>>>>> 
>>>>>>> And should be covered with the —force parameter we added.
>>>>>>> 
>>>>>>> What do you think?
>>>>>>> 
>>>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
>>>> написал(а):
>>>>>>>> 
>>>>>>>>    Hi, all.
>>>>>>>> 
>>>>>>>> Fixes for control.sh and the REST have been merged. Could anyone
>> take
>>>> a look to the previous email with an issue? Isn't this conductvery
>> wierd?
>>>>>>>> 
>>>> 
>> 
>> 
> 
> -- 
> 
> Best regards,
> Alexei Scherbakov

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexei Scherbakov <al...@gmail.com>]

Nikolay,

I'm on the same page with Ivan.

Having "force" flag in public API as preposterous as having it in
System.exit.
For me it looks like badly designed API.
If a call to some method is dangerous it should be clearly specified in the
javadoc.
I'm also against some "temporary" API.

We should:

1. Partially remove IGNITE-12701 except javadoc part. Note control.sh for a
long time has support for a confirmation on deactivation (interactive mode).
2. IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true already preserves memory content
after deactivation. We should start working on restoring page memory state
after subsequent reactivation.
3. Describe current behavior for in-memory cache on deactivation in Ignite
documentation.


пн, 23 мар. 2020 г. в 21:22, Nikolay Izhikov <ni...@apache.org>:

> Hello, Ivan.
>
> > Seems like we don't have a final agreement on whether we should add force
> flag to deactivation API.
>
> I think the only question is - Do we need —force flag in Java API or not.
>
>
> > As a final solution, I'd want to see behavior when all in-memory data is
> available after deactivation and further activation.
>
> Agree.
>
> > I believe it’s possible to don't deallocate memory
> > (like mentioned before, we already can achieve that with
> IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded data
> pages on next activation and caches start.
>
> As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
> Can you provide a simple reproducer when in-memory data not cleared on
> deactivation?
>
> > Considering this, do we really need to introduce force flag as a
> temporary precaution?
>
> My answer is yes we need it.
> Right now, we can’t prevent data loss on deactivation for in-memory caches.
>
> For me, the ultimate value of Ignite into real production environment is
> user data.
> If we have some cases when data is lost - we should avoid it as hard as we
> can.
>
> So, for now, this flag required.
>
> > I suggest to rollback [2] from AI master, stop working on [1] and focus
> on how to implement keeping in-memory data after the deactivation.
>
> I think we can remove —force flag only after implementation of keeping
> in-memory data on deactivation would be finished.
> If that happens before 2.9 then we can have clearer API.
>
> > 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com>
> написал(а):
> >
> > Folks,
> >
> > Let's revive this discussion until it's too late and all API changes are
> > merged to master [1].
> > Seems like we don't have a final agreement on whether we should add force
> > flag to deactivation API.
> >
> > First of all, I think we are all on the same page that in-memory caches
> > data vanishing on deactivation is counter-intuitive and dangerous. As a
> > final solution, I'd want to see behavior when all in-memory data is
> > available after deactivation and further activation. I believe it's
> > possible to don't deallocate memory (like mentioned before, we already
> can
> > achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
> > reuse all loaded data pages on next activation and caches start.
> >
> > Also, this is a wider question, but: do we understand what cluster
> > deactivation is actually intended for? I can only think of two cases:
> > - graceful cluster shutdown: an ability to cut checkpoints and to end
> > transactional load consistently prior to further stop of all nodes
> > - blocking all API (both reads and writes) due to some maintenance
> > Neither of them requires forcefully clearing all in-memory data on
> > deactivation. If everyone agrees, from now on we should assume data
> > clearing as system design flaw that should be fixed, not as possible
> > scenario which we should support on the API level.
> >
> > Considering this, do we really need to introduce force flag as a
> temporary
> > precaution? I have at least two reasons against it:
> > 1) Once API was changed and released, we have to support it until the
> next
> > major release. If we all understand that data vanishing issue is
> fixable, I
> > believe we shouldn't engrave in the API flags that will become pointless.
> > 2) More personal one, but I'm against any force flags in the API. This
> > makes API harder to understand; more than that, the presence of such
> flags
> > just highlights that API is poorly designed.
> >
> > I suggest to rollback [2] from AI master, stop working on [1] and focus
> on
> > how to implement keeping in-memory data after the deactivation.
> > I think we can still require user consent for deactivation via control.sh
> > (it already requires --yes) and JMX.
> >
> > Thoughts?
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-12614
> > [2]: https://issues.apache.org/jira/browse/IGNITE-12701
> >
> > --
> > Ivan
> >
> >
> > On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com>
> wrote:
> >
> >> Nikolay, I think we should reconsider clearing at least system caches
> >> when deactivating.
> >>
> >> 17.03.2020 14:18, Nikolay Izhikov пишет:
> >>> Hello, Vladimir.
> >>>
> >>> I don’t get it.
> >>>
> >>> What is your proposal?
> >>> What we should do?
> >>>
> >>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
> >> написал(а):
> >>>>
> >>>> Nikolay, hi.
> >>>>
> >>>>>>> And should be covered with the  —force parameter we added.
> >>>> As fix for user cases - yes. My idea is to emphasize overall ability
> to
> >> lose various objects, not only data. Probably might be reconsidered in
> >> future.
> >>>>
> >>>>
> >>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
> >>>>> Hello, Vladimir.
> >>>>>
> >>>>> If there is at lease one persistent data region then system data
> >> region also becomes persistent.
> >>>>> Your example applies only to pure in-memory clusters.
> >>>>>
> >>>>> And should be covered with the —force parameter we added.
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
> >> написал(а):
> >>>>>>
> >>>>>>     Hi, all.
> >>>>>>
> >>>>>> Fixes for control.sh and the REST have been merged. Could anyone
> take
> >> a look to the previous email with an issue? Isn't this conductvery
> wierd?
> >>>>>>
> >>
>
>

-- 

Best regards,
Alexei Scherbakov

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Ivan.

> Seems like we don't have a final agreement on whether we should add force
flag to deactivation API.

I think the only question is - Do we need —force flag in Java API or not.


> As a final solution, I'd want to see behavior when all in-memory data is available after deactivation and further activation. 

Agree.

> I believe it’s possible to don't deallocate memory 
> (like mentioned before, we already can achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully reuse all loaded data pages on next activation and caches start.

As far as I know, IGNITE_REUSE_MEMORY_ON_DEACTIVATE is for *other* purpose.
Can you provide a simple reproducer when in-memory data not cleared on deactivation?

> Considering this, do we really need to introduce force flag as a temporary precaution?

My answer is yes we need it.
Right now, we can’t prevent data loss on deactivation for in-memory caches.

For me, the ultimate value of Ignite into real production environment is user data.
If we have some cases when data is lost - we should avoid it as hard as we can.

So, for now, this flag required.

> I suggest to rollback [2] from AI master, stop working on [1] and focus on how to implement keeping in-memory data after the deactivation.

I think we can remove —force flag only after implementation of keeping in-memory data on deactivation would be finished.
If that happens before 2.9 then we can have clearer API.

> 23 марта 2020 г., в 18:47, Ivan Rakov <iv...@gmail.com> написал(а):
> 
> Folks,
> 
> Let's revive this discussion until it's too late and all API changes are
> merged to master [1].
> Seems like we don't have a final agreement on whether we should add force
> flag to deactivation API.
> 
> First of all, I think we are all on the same page that in-memory caches
> data vanishing on deactivation is counter-intuitive and dangerous. As a
> final solution, I'd want to see behavior when all in-memory data is
> available after deactivation and further activation. I believe it's
> possible to don't deallocate memory (like mentioned before, we already can
> achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
> reuse all loaded data pages on next activation and caches start.
> 
> Also, this is a wider question, but: do we understand what cluster
> deactivation is actually intended for? I can only think of two cases:
> - graceful cluster shutdown: an ability to cut checkpoints and to end
> transactional load consistently prior to further stop of all nodes
> - blocking all API (both reads and writes) due to some maintenance
> Neither of them requires forcefully clearing all in-memory data on
> deactivation. If everyone agrees, from now on we should assume data
> clearing as system design flaw that should be fixed, not as possible
> scenario which we should support on the API level.
> 
> Considering this, do we really need to introduce force flag as a temporary
> precaution? I have at least two reasons against it:
> 1) Once API was changed and released, we have to support it until the next
> major release. If we all understand that data vanishing issue is fixable, I
> believe we shouldn't engrave in the API flags that will become pointless.
> 2) More personal one, but I'm against any force flags in the API. This
> makes API harder to understand; more than that, the presence of such flags
> just highlights that API is poorly designed.
> 
> I suggest to rollback [2] from AI master, stop working on [1] and focus on
> how to implement keeping in-memory data after the deactivation.
> I think we can still require user consent for deactivation via control.sh
> (it already requires --yes) and JMX.
> 
> Thoughts?
> 
> [1]: https://issues.apache.org/jira/browse/IGNITE-12614
> [2]: https://issues.apache.org/jira/browse/IGNITE-12701
> 
> --
> Ivan
> 
> 
> On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com> wrote:
> 
>> Nikolay, I think we should reconsider clearing at least system caches
>> when deactivating.
>> 
>> 17.03.2020 14:18, Nikolay Izhikov пишет:
>>> Hello, Vladimir.
>>> 
>>> I don’t get it.
>>> 
>>> What is your proposal?
>>> What we should do?
>>> 
>>>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
>> написал(а):
>>>> 
>>>> Nikolay, hi.
>>>> 
>>>>>>> And should be covered with the  —force parameter we added.
>>>> As fix for user cases - yes. My idea is to emphasize overall ability to
>> lose various objects, not only data. Probably might be reconsidered in
>> future.
>>>> 
>>>> 
>>>> 17.03.2020 13:49, Nikolay Izhikov пишет:
>>>>> Hello, Vladimir.
>>>>> 
>>>>> If there is at lease one persistent data region then system data
>> region also becomes persistent.
>>>>> Your example applies only to pure in-memory clusters.
>>>>> 
>>>>> And should be covered with the —force parameter we added.
>>>>> 
>>>>> What do you think?
>>>>> 
>>>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
>> написал(а):
>>>>>> 
>>>>>>     Hi, all.
>>>>>> 
>>>>>> Fixes for control.sh and the REST have been merged. Could anyone take
>> a look to the previous email with an issue? Isn't this conductvery wierd?
>>>>>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Rakov <iv...@gmail.com>]

Folks,

Let's revive this discussion until it's too late and all API changes are
merged to master [1].
Seems like we don't have a final agreement on whether we should add force
flag to deactivation API.

First of all, I think we are all on the same page that in-memory caches
data vanishing on deactivation is counter-intuitive and dangerous. As a
final solution, I'd want to see behavior when all in-memory data is
available after deactivation and further activation. I believe it's
possible to don't deallocate memory (like mentioned before, we already can
achieve that with IGNITE_REUSE_MEMORY_ON_DEACTIVATE=true) and carefully
reuse all loaded data pages on next activation and caches start.

Also, this is a wider question, but: do we understand what cluster
deactivation is actually intended for? I can only think of two cases:
- graceful cluster shutdown: an ability to cut checkpoints and to end
transactional load consistently prior to further stop of all nodes
- blocking all API (both reads and writes) due to some maintenance
Neither of them requires forcefully clearing all in-memory data on
deactivation. If everyone agrees, from now on we should assume data
clearing as system design flaw that should be fixed, not as possible
scenario which we should support on the API level.

Considering this, do we really need to introduce force flag as a temporary
precaution? I have at least two reasons against it:
1) Once API was changed and released, we have to support it until the next
major release. If we all understand that data vanishing issue is fixable, I
believe we shouldn't engrave in the API flags that will become pointless.
2) More personal one, but I'm against any force flags in the API. This
makes API harder to understand; more than that, the presence of such flags
just highlights that API is poorly designed.

I suggest to rollback [2] from AI master, stop working on [1] and focus on
how to implement keeping in-memory data after the deactivation.
I think we can still require user consent for deactivation via control.sh
(it already requires --yes) and JMX.

Thoughts?

[1]: https://issues.apache.org/jira/browse/IGNITE-12614
[2]: https://issues.apache.org/jira/browse/IGNITE-12701

--
Ivan


On Tue, Mar 17, 2020 at 2:26 PM Vladimir Steshin <vl...@gmail.com> wrote:

> Nikolay, I think we should reconsider clearing at least system caches
> when deactivating.
>
> 17.03.2020 14:18, Nikolay Izhikov пишет:
> > Hello, Vladimir.
> >
> > I don’t get it.
> >
> > What is your proposal?
> > What we should do?
> >
> >> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com>
> написал(а):
> >>
> >> Nikolay, hi.
> >>
> >>>>> And should be covered with the  —force parameter we added.
> >> As fix for user cases - yes. My idea is to emphasize overall ability to
> lose various objects, not only data. Probably might be reconsidered in
> future.
> >>
> >>
> >> 17.03.2020 13:49, Nikolay Izhikov пишет:
> >>> Hello, Vladimir.
> >>>
> >>> If there is at lease one persistent data region then system data
> region also becomes persistent.
> >>> Your example applies only to pure in-memory clusters.
> >>>
> >>> And should be covered with the —force parameter we added.
> >>>
> >>> What do you think?
> >>>
> >>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com>
> написал(а):
> >>>>
> >>>>      Hi, all.
> >>>>
> >>>> Fixes for control.sh and the REST have been merged. Could anyone take
> a look to the previous email with an issue? Isn't this conductvery wierd?
> >>>>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Nikolay, I think we should reconsider clearing at least system caches 
when deactivating.

17.03.2020 14:18, Nikolay Izhikov пишет:
> Hello, Vladimir.
>
> I don’t get it.
>
> What is your proposal?
> What we should do?
>
>> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com> написал(а):
>>
>> Nikolay, hi.
>>
>>>>> And should be covered with the  —force parameter we added.
>> As fix for user cases - yes. My idea is to emphasize overall ability to lose various objects, not only data. Probably might be reconsidered in future.
>>
>>
>> 17.03.2020 13:49, Nikolay Izhikov пишет:
>>> Hello, Vladimir.
>>>
>>> If there is at lease one persistent data region then system data region also becomes persistent.
>>> Your example applies only to pure in-memory clusters.
>>>
>>> And should be covered with the —force parameter we added.
>>>
>>> What do you think?
>>>
>>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com> написал(а):
>>>>
>>>>      Hi, all.
>>>>
>>>> Fixes for control.sh and the REST have been merged. Could anyone take a look to the previous email with an issue? Isn't this conductvery wierd?
>>>>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Vladimir.

I don’t get it.

What is your proposal?
What we should do?

> 17 марта 2020 г., в 14:11, Vladimir Steshin <vl...@gmail.com> написал(а):
> 
> Nikolay, hi.
> 
>>>> And should be covered with the  —force parameter we added.
> 
> As fix for user cases - yes. My idea is to emphasize overall ability to lose various objects, not only data. Probably might be reconsidered in future.
> 
> 
> 17.03.2020 13:49, Nikolay Izhikov пишет:
>> Hello, Vladimir.
>> 
>> If there is at lease one persistent data region then system data region also becomes persistent.
>> Your example applies only to pure in-memory clusters.
>> 
>> And should be covered with the —force parameter we added.
>> 
>> What do you think?
>> 
>>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com> написал(а):
>>> 
>>>     Hi, all.
>>> 
>>> Fixes for control.sh and the REST have been merged. Could anyone take a look to the previous email with an issue? Isn't this conductvery wierd?
>>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Nikolay, hi.

>>> And should be covered with the  —force parameter we added.

As fix for user cases - yes. My idea is to emphasize overall ability to 
lose various objects, not only data. Probably might be reconsidered in 
future.


17.03.2020 13:49, Nikolay Izhikov пишет:
> Hello, Vladimir.
>
> If there is at lease one persistent data region then system data region also becomes persistent.
> Your example applies only to pure in-memory clusters.
>
> And should be covered with the —force parameter we added.
>
> What do you think?
>
>> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com> написал(а):
>>
>>      Hi, all.
>>
>> Fixes for control.sh and the REST have been merged. Could anyone take a look to the previous email with an issue? Isn't this conductvery wierd?
>>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Vladimir.

If there is at lease one persistent data region then system data region also becomes persistent.
Your example applies only to pure in-memory clusters.

And should be covered with the —force parameter we added.

What do you think?

> 17 марта 2020 г., в 13:45, Vladimir Steshin <vl...@gmail.com> написал(а):
> 
>     Hi, all.
> 
> Fixes for control.sh and the REST have been merged. Could anyone take a look to the previous email with an issue? Isn't this conductvery wierd?
> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

     Hi, all.

Fixes for control.sh and the REST have been merged. Could anyone take a 
look to the previous email with an issue? Isn't this conductvery wierd?

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hi, Igniters.

I've found other deactivation issue. Should we consider this behavior as 
a bug?

IgniteEx ignite = startGrids(3);

         IgniteAtomicLong atomicLong = ignite.atomicLong("atomic", 10L, 
true);


         IgniteLock lock = ignite.reentrantLock("lock", true, false, true);


         lock.lock();

         assertEquals(10L, atomicLong.get());
         assertTrue(lock.isLocked());

         lock.unlock();

         ignite.active(false);
         ignite.active(true);

         // Failed: java.lang.NullPointerException at 
GridCacheLockImpl.java:496
         assertFalse(lock.isLocked());

         // Failrd: org.apache.ignite.IgniteException: Failed to find 
atomic long: testAtomic
         assertEquals(10L, atomicLong.get());

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Alexey.

> We need to agree whether IgniteMxBean#active(boolean) and Ignite#active(boolean) behaving differently is ok from the user side.

I think it’s not OK.

We should provide consistent behavior for all management APIs: jmx, control.sh, REST, Java API.
The primary use-case I keep in mind - third party management console written in java.
It will use Java API and should have an ability to execute «safe»(without data vanish) and «unsafe» deactivation.


> 18 февр. 2020 г., в 12:28, Alexey Goncharuk <al...@gmail.com> написал(а):
> 
> I do not think the change size can even be an argument for not doing a
> proper improvement. We need to agree whether IgniteMxBean#active(boolean)
> and Ignite#active(boolean) behaving differently is ok from the user side.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

I do not think the change size can even be an argument for not doing a
proper improvement. We need to agree whether IgniteMxBean#active(boolean)
and Ignite#active(boolean) behaving differently is ok from the user side.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Alexey, as I mentioned, at now we have same implementation for IgniteMXBean
and Ignite in IgniteKernal. Already is. I agree, the implementations should
be decoupled. But extracting IgniteMXBean from IgniteKernal seems too bulky
solution to me. Tacking in account lot of variants of deactivation methods
(many deprecated) and invocation points I propose to bring single
centralized solution.

пн, 17 февр. 2020 г. в 16:59, Alexey Goncharuk <al...@gmail.com>:

> пн, 17 февр. 2020 г. в 14:18, Vladimir Steshin <vl...@gmail.com>:
>
> > > The only reason is the same implementation IgniteMXBean#active(boolean)
> > > and Ignite#active(boolean). Looks like we have to choose whether to
> allow
> > > user erase data unexpectedly or to change behavior of the API call.
> >
>
> Vladimir, those two interfaces (IgniteMXBean and Ignite) are decoupled and
> must not share the same implementation. Internally, we can route the
> methods to separate implementations. Or am I missing something?
>
> For me the analogy is pretty simple - we do have to specify an additional
> flag for 'rm' command, but we do not have any confirmation flags for
> syscalls.
>
> --AG
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

пн, 17 февр. 2020 г. в 14:18, Vladimir Steshin <vl...@gmail.com>:

> > The only reason is the same implementation IgniteMXBean#active(boolean)
> > and Ignite#active(boolean). Looks like we have to choose whether to allow
> > user erase data unexpectedly or to change behavior of the API call.
>

Vladimir, those two interfaces (IgniteMXBean and Ignite) are decoupled and
must not share the same implementation. Internally, we can route the
methods to separate implementations. Or am I missing something?

For me the analogy is pretty simple - we do have to specify an additional
flag for 'rm' command, but we do not have any confirmation flags for
syscalls.

--AG

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

>
> Hi,  Vyacheslav.
>
>
>
> >>> In general, it is a caching layer above an existing 3rd party database.
> That is all.
> >>> I don't think so. It does not really matter which persistence is used
> (Ignite native persistence or RDBMS).
>
>
>
> Ok. I think we might also check if read/write-through is enabled for a
> cache to consider it as persistent and not to warn user, not to require
> ‘force’ flag.
>
>
>
>
>
> >>> it does make sense to me (control utility and jmx) To be honest, I
> don't
> see a reason to introduce this flag into Java API.
>
>
>
> The only reason is the same implementation IgniteMXBean#active(boolean)
> and Ignite#active(boolean). Looks like we have to choose whether to allow
> user erase data unexpectedly or to change behavior of the API call.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hi Nikolay,

> 1. Explicitly warn user about consequences of the deactivation.
I absolutely agree it must be mentioned in the Javadoc, documentation, and
etc.

> Require additional confirmation.
it does make sense to me (control utility and jmx) To be honest, I don't
see a reason to introduce this flag into Java API.

Thanks,
Slava.


пн, 17 февр. 2020 г. в 12:57, Nikolay Izhikov <ni...@apache.org>:

> Hello, Vyacheslav.
>
> > In my humble opinion, until we have a consensus that the existing
> behavior is a bug (and that is the question for now)
>
>
> I think we have consensus that current behavior is *NOT A BUG*.
> It is a design choice.
>
> But, we have consensus that consequences of this choice is not transparent
> to the end-user.
> So we have:
>
> 1. Explicitly warn user about consequences of the deactivation.
> 2. Require additional confirmation.
>
>
>
> > 17 февр. 2020 г., в 12:38, Вячеслав Коптилин <sl...@gmail.com>
> написал(а):
> >
> > Hello Vladimir,
> >
> >> We do not know what exactly does *3-rd party persistence*.
> > From my point of view, this is clearly stated here
> > https://www.jcp.org/en/jsr/detail?id=107 |
> > https://github.com/jsr107/jsr107spec and here
> > https://apacheignite.readme.io/docs/3rd-party-store.
> > In general, it is a caching layer above an existing 3rd party database.
> > That is all.
> >
> >> Such caches might be considered as in-memory only.
> > I don't think so. It does not really matter which persistence is used
> > (Ignite native persistence or RDBMS).
> >
> > I would like to highlight the following: In my humble opinion, until we
> > have a consensus that the existing behavior is a bug (and that is the
> > question for now) we should not change it.
> > As mentioned by Alex G: "Inactive state deallocates all possible
> resources
> > by design, including the data regions." Perhaps, this is not a good
> > approach, but...
> >
> > Thanks,
> > Slava.
> >
> > пт, 14 февр. 2020 г. в 18:56, Vladimir Steshin <vl...@gmail.com>:
> >
> >> Vyacheslav,
> >>
> >>
> >>
> >>>>> I am talking about MIXED cluster with persistent cache and
> *in-memory*
> >> cache which is backed by *3-rd party persistence*.
> >>
> >>
> >>
> >> We do not know what exactly does *3-rd party persistence*. It may store
> >> only specific filtered data, small part of data. I think it is out of
> >> responsibility of the persistence. Such caches might be considered as
> >> in-memory only.
> >>
> >>
> >>
> >>>>> That is why I do not like to expose such functionality through JMX.
> >>
> >> But it is exposed also in CLI and REST. Just various call types. Why
> hide
> >> it from JMX? Or why it is supposed to act differently? If CLI (and REST)
> >> requires forced deactivation, why JMX would act in other way at the same
> >> conditions?
> >>
> >> пт, 14 февр. 2020 г. в 18:40, Вячеслав Коптилин <
> slava.koptilin@gmail.com
> >>> :
> >>
> >>> Hi Vladimir,
> >>>
> >>>> if you have only persistent caches, no warning/confirmation is
> supposed
> >>> at all.
> >>> I am talking about MIXED cluster with persistent cache and *in-memory*
> >>> cache which is backed by *3-rd party persistence*.
> >>>
> >>>> I’m afraid this won’t stop anyone from using old deprecated
> >>> IgniteMXBean#active(boolean)
> >>> That is why I do not like to expose such functionality through JMX.
> >>>
> >>> Thanks,
> >>> S.
> >>>
> >>> пт, 14 февр. 2020 г. в 18:02, Vladimir Steshin <vl...@gmail.com>:
> >>>
> >>>> Vyacheslav,
> >>>>
> >>>>>>> Let's assume that I have a mixed cluster with persistent cache and
> >>>> in-memory cache which is backed by 3-rd party persistence. I see no
> >>> reason
> >>>> to throw an exception in that case at least.
> >>>>
> >>>>
> >>>>
> >>>> if you have only persistent caches, no warning/confirmation is
> supposed
> >>> at
> >>>> all.
> >>>>
> >>>>
> >>>>
> >>>>>>> Is it possible to
> >>>> add new methods as follows: activateCluster()/deactivateCluster() and
> >>>> deprecate IgniteMXBean#active(boolean)?
> >>>>
> >>>>
> >>>>
> >>>> I’m afraid this won’t stop anyone from using old deprecated
> >>>> IgniteMXBean#active(boolean).
> >>>> It is quite obvious to execute through JMX despite it is deprecated.
> >>>>
> >>>> пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <
> >>> slava.koptilin@gmail.com
> >>>>> :
> >>>>
> >>>>> Hello Nikolay,
> >>>>>
> >>>>>> Should public java API continue to silently clear in-memory caches?
> >>>>> Let's assume that I have a mixed cluster with persistent cache and
> >>>>> in-memory cache which is backed by 3-rd party persistence. I see no
> >>>> reason
> >>>>> to throw an exception in that case at least.
> >>>>> Anyway, this fact should be clearly stated in the Javadoc and
> >>>> documentation
> >>>>> of course.
> >>>>>
> >>>>>> What is your suggestion for the API?
> >>>>> I think we are talking about JMX methods. Am I correct? Is it
> >> possible
> >>> to
> >>>>> add new methods as follows: activateCluster()/deactivateCluster() and
> >>>>> deprecate IgniteMXBean#active(boolean)?
> >>>>> Does this make sense? Am I missing something?
> >>>>>
> >>>>> Thanks,
> >>>>> S.
> >>>>>
> >>>>> пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
> >>>>>
> >>>>>> Vyacheslav.
> >>>>>>
> >>>>>> What is your suggestion for the API?
> >>>>>>
> >>>>>> Single implementation for both Ignite#active(boolean) and
> >>>>>> IgniteMXBean#active(boolean)
> >>>>>> Should public java API continue to silently clears in-memory
> >> caches?
> >>>>>>
> >>>>>>
> >>>>>>> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <
> >>>> slava.koptilin@gmail.com
> >>>>>>
> >>>>>> написал(а):
> >>>>>>>
> >>>>>>> Hello Vladimir,
> >>>>>>>
> >>>>>>>> adding a new method with force flag means old methods change
> >> their
> >>>>>>> behavior:
> >>>>>>> I don't think that changing the behavior of public API is the
> >> right
> >>>>> way.
> >>>>>>> Moreover, I agree with Alex that there is no need to introduce a
> >>>>>>> "confirmation" flag to the java API.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> S.
> >>>>>>>
> >>>>>>> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <
> >> vladsz83@gmail.com
> >>>> :
> >>>>>>>
> >>>>>>>> Alexey, adding a new method with force flag means old methods
> >>> change
> >>>>>> their
> >>>>>>>> behavior: they are considered as executed without ‘force‘ flag
> >> and
> >>>> can
> >>>>>> fail
> >>>>>>>> to prevent data loss. Ignite and IgniteMXBean are different
> >>>>> interfaces.
> >>>>>>>> Unfortunately, they have same method
> >>>>>>>>
> >>>>>>>> void  active(boolean active)
> >>>>>>>>
> >>>>>>>> When executed as IgniteMXBean it should fail if user can lose
> >>> data.
> >>>>> When
> >>>>>>>> executed from code via interface Ignite probably not. To solve
> >>> this
> >>>> I
> >>>>>>>> suggest to add ‘force’ flag for every deactivation mode:
> >>>> CLI/JMX/REST
> >>>>>> and
> >>>>>>>> other API.
> >>>>>>>>
> >>>>>>>> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>> :
> >>>>>>>>
> >>>>>>>>> Igniters,
> >>>>>>>>>
> >>>>>>>>> Do we really need the confirmation flag on the public API? I
> >>>>> absolutely
> >>>>>>>>> agree on the CLI and MXBean, but what is the reason for the
> >> flag
> >>> in
> >>>>> the
> >>>>>>>>> API? It will be specified at the compile time anyway and does
> >> not
> >>>>>> prevent
> >>>>>>>>> any user error.
> >>>>>>>>> From the implementation point of view I see no contradiction -
> >> we
> >>>> can
> >>>>>> add
> >>>>>>>>> the new method to the MXBean, but nothing forces us to add it
> >> to
> >>>>> Ignite
> >>>>>>>>> interface - those interfaces are not related.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Vyacheslav.

> In my humble opinion, until we have a consensus that the existing behavior is a bug (and that is the question for now)


I think we have consensus that current behavior is *NOT A BUG*.
It is a design choice.

But, we have consensus that consequences of this choice is not transparent to the end-user.
So we have:

1. Explicitly warn user about consequences of the deactivation.
2. Require additional confirmation.



> 17 февр. 2020 г., в 12:38, Вячеслав Коптилин <sl...@gmail.com> написал(а):
> 
> Hello Vladimir,
> 
>> We do not know what exactly does *3-rd party persistence*.
> From my point of view, this is clearly stated here
> https://www.jcp.org/en/jsr/detail?id=107 |
> https://github.com/jsr107/jsr107spec and here
> https://apacheignite.readme.io/docs/3rd-party-store.
> In general, it is a caching layer above an existing 3rd party database.
> That is all.
> 
>> Such caches might be considered as in-memory only.
> I don't think so. It does not really matter which persistence is used
> (Ignite native persistence or RDBMS).
> 
> I would like to highlight the following: In my humble opinion, until we
> have a consensus that the existing behavior is a bug (and that is the
> question for now) we should not change it.
> As mentioned by Alex G: "Inactive state deallocates all possible resources
> by design, including the data regions." Perhaps, this is not a good
> approach, but...
> 
> Thanks,
> Slava.
> 
> пт, 14 февр. 2020 г. в 18:56, Vladimir Steshin <vl...@gmail.com>:
> 
>> Vyacheslav,
>> 
>> 
>> 
>>>>> I am talking about MIXED cluster with persistent cache and *in-memory*
>> cache which is backed by *3-rd party persistence*.
>> 
>> 
>> 
>> We do not know what exactly does *3-rd party persistence*. It may store
>> only specific filtered data, small part of data. I think it is out of
>> responsibility of the persistence. Such caches might be considered as
>> in-memory only.
>> 
>> 
>> 
>>>>> That is why I do not like to expose such functionality through JMX.
>> 
>> But it is exposed also in CLI and REST. Just various call types. Why hide
>> it from JMX? Or why it is supposed to act differently? If CLI (and REST)
>> requires forced deactivation, why JMX would act in other way at the same
>> conditions?
>> 
>> пт, 14 февр. 2020 г. в 18:40, Вячеслав Коптилин <slava.koptilin@gmail.com
>>> :
>> 
>>> Hi Vladimir,
>>> 
>>>> if you have only persistent caches, no warning/confirmation is supposed
>>> at all.
>>> I am talking about MIXED cluster with persistent cache and *in-memory*
>>> cache which is backed by *3-rd party persistence*.
>>> 
>>>> I’m afraid this won’t stop anyone from using old deprecated
>>> IgniteMXBean#active(boolean)
>>> That is why I do not like to expose such functionality through JMX.
>>> 
>>> Thanks,
>>> S.
>>> 
>>> пт, 14 февр. 2020 г. в 18:02, Vladimir Steshin <vl...@gmail.com>:
>>> 
>>>> Vyacheslav,
>>>> 
>>>>>>> Let's assume that I have a mixed cluster with persistent cache and
>>>> in-memory cache which is backed by 3-rd party persistence. I see no
>>> reason
>>>> to throw an exception in that case at least.
>>>> 
>>>> 
>>>> 
>>>> if you have only persistent caches, no warning/confirmation is supposed
>>> at
>>>> all.
>>>> 
>>>> 
>>>> 
>>>>>>> Is it possible to
>>>> add new methods as follows: activateCluster()/deactivateCluster() and
>>>> deprecate IgniteMXBean#active(boolean)?
>>>> 
>>>> 
>>>> 
>>>> I’m afraid this won’t stop anyone from using old deprecated
>>>> IgniteMXBean#active(boolean).
>>>> It is quite obvious to execute through JMX despite it is deprecated.
>>>> 
>>>> пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <
>>> slava.koptilin@gmail.com
>>>>> :
>>>> 
>>>>> Hello Nikolay,
>>>>> 
>>>>>> Should public java API continue to silently clear in-memory caches?
>>>>> Let's assume that I have a mixed cluster with persistent cache and
>>>>> in-memory cache which is backed by 3-rd party persistence. I see no
>>>> reason
>>>>> to throw an exception in that case at least.
>>>>> Anyway, this fact should be clearly stated in the Javadoc and
>>>> documentation
>>>>> of course.
>>>>> 
>>>>>> What is your suggestion for the API?
>>>>> I think we are talking about JMX methods. Am I correct? Is it
>> possible
>>> to
>>>>> add new methods as follows: activateCluster()/deactivateCluster() and
>>>>> deprecate IgniteMXBean#active(boolean)?
>>>>> Does this make sense? Am I missing something?
>>>>> 
>>>>> Thanks,
>>>>> S.
>>>>> 
>>>>> пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
>>>>> 
>>>>>> Vyacheslav.
>>>>>> 
>>>>>> What is your suggestion for the API?
>>>>>> 
>>>>>> Single implementation for both Ignite#active(boolean) and
>>>>>> IgniteMXBean#active(boolean)
>>>>>> Should public java API continue to silently clears in-memory
>> caches?
>>>>>> 
>>>>>> 
>>>>>>> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <
>>>> slava.koptilin@gmail.com
>>>>>> 
>>>>>> написал(а):
>>>>>>> 
>>>>>>> Hello Vladimir,
>>>>>>> 
>>>>>>>> adding a new method with force flag means old methods change
>> their
>>>>>>> behavior:
>>>>>>> I don't think that changing the behavior of public API is the
>> right
>>>>> way.
>>>>>>> Moreover, I agree with Alex that there is no need to introduce a
>>>>>>> "confirmation" flag to the java API.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> S.
>>>>>>> 
>>>>>>> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <
>> vladsz83@gmail.com
>>>> :
>>>>>>> 
>>>>>>>> Alexey, adding a new method with force flag means old methods
>>> change
>>>>>> their
>>>>>>>> behavior: they are considered as executed without ‘force‘ flag
>> and
>>>> can
>>>>>> fail
>>>>>>>> to prevent data loss. Ignite and IgniteMXBean are different
>>>>> interfaces.
>>>>>>>> Unfortunately, they have same method
>>>>>>>> 
>>>>>>>> void  active(boolean active)
>>>>>>>> 
>>>>>>>> When executed as IgniteMXBean it should fail if user can lose
>>> data.
>>>>> When
>>>>>>>> executed from code via interface Ignite probably not. To solve
>>> this
>>>> I
>>>>>>>> suggest to add ‘force’ flag for every deactivation mode:
>>>> CLI/JMX/REST
>>>>>> and
>>>>>>>> other API.
>>>>>>>> 
>>>>>>>> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>> :
>>>>>>>> 
>>>>>>>>> Igniters,
>>>>>>>>> 
>>>>>>>>> Do we really need the confirmation flag on the public API? I
>>>>> absolutely
>>>>>>>>> agree on the CLI and MXBean, but what is the reason for the
>> flag
>>> in
>>>>> the
>>>>>>>>> API? It will be specified at the compile time anyway and does
>> not
>>>>>> prevent
>>>>>>>>> any user error.
>>>>>>>>> From the implementation point of view I see no contradiction -
>> we
>>>> can
>>>>>> add
>>>>>>>>> the new method to the MXBean, but nothing forces us to add it
>> to
>>>>> Ignite
>>>>>>>>> interface - those interfaces are not related.
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hello Vladimir,

> We do not know what exactly does *3-rd party persistence*.
From my point of view, this is clearly stated here
https://www.jcp.org/en/jsr/detail?id=107 |
https://github.com/jsr107/jsr107spec and here
https://apacheignite.readme.io/docs/3rd-party-store.
In general, it is a caching layer above an existing 3rd party database.
That is all.

> Such caches might be considered as in-memory only.
I don't think so. It does not really matter which persistence is used
(Ignite native persistence or RDBMS).

I would like to highlight the following: In my humble opinion, until we
have a consensus that the existing behavior is a bug (and that is the
question for now) we should not change it.
As mentioned by Alex G: "Inactive state deallocates all possible resources
by design, including the data regions." Perhaps, this is not a good
approach, but...

Thanks,
Slava.

пт, 14 февр. 2020 г. в 18:56, Vladimir Steshin <vl...@gmail.com>:

> Vyacheslav,
>
>
>
> >>> I am talking about MIXED cluster with persistent cache and *in-memory*
> cache which is backed by *3-rd party persistence*.
>
>
>
> We do not know what exactly does *3-rd party persistence*. It may store
> only specific filtered data, small part of data. I think it is out of
> responsibility of the persistence. Such caches might be considered as
> in-memory only.
>
>
>
> >>> That is why I do not like to expose such functionality through JMX.
>
> But it is exposed also in CLI and REST. Just various call types. Why hide
> it from JMX? Or why it is supposed to act differently? If CLI (and REST)
> requires forced deactivation, why JMX would act in other way at the same
> conditions?
>
> пт, 14 февр. 2020 г. в 18:40, Вячеслав Коптилин <slava.koptilin@gmail.com
> >:
>
> > Hi Vladimir,
> >
> > > if you have only persistent caches, no warning/confirmation is supposed
> > at all.
> > I am talking about MIXED cluster with persistent cache and *in-memory*
> > cache which is backed by *3-rd party persistence*.
> >
> > > I’m afraid this won’t stop anyone from using old deprecated
> > IgniteMXBean#active(boolean)
> > That is why I do not like to expose such functionality through JMX.
> >
> > Thanks,
> > S.
> >
> > пт, 14 февр. 2020 г. в 18:02, Vladimir Steshin <vl...@gmail.com>:
> >
> > >  Vyacheslav,
> > >
> > > >>> Let's assume that I have a mixed cluster with persistent cache and
> > > in-memory cache which is backed by 3-rd party persistence. I see no
> > reason
> > > to throw an exception in that case at least.
> > >
> > >
> > >
> > > if you have only persistent caches, no warning/confirmation is supposed
> > at
> > > all.
> > >
> > >
> > >
> > > >>> Is it possible to
> > > add new methods as follows: activateCluster()/deactivateCluster() and
> > > deprecate IgniteMXBean#active(boolean)?
> > >
> > >
> > >
> > > I’m afraid this won’t stop anyone from using old deprecated
> > > IgniteMXBean#active(boolean).
> > > It is quite obvious to execute through JMX despite it is deprecated.
> > >
> > > пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <
> > slava.koptilin@gmail.com
> > > >:
> > >
> > > > Hello Nikolay,
> > > >
> > > > > Should public java API continue to silently clear in-memory caches?
> > > > Let's assume that I have a mixed cluster with persistent cache and
> > > > in-memory cache which is backed by 3-rd party persistence. I see no
> > > reason
> > > > to throw an exception in that case at least.
> > > > Anyway, this fact should be clearly stated in the Javadoc and
> > > documentation
> > > > of course.
> > > >
> > > > > What is your suggestion for the API?
> > > > I think we are talking about JMX methods. Am I correct? Is it
> possible
> > to
> > > > add new methods as follows: activateCluster()/deactivateCluster() and
> > > > deprecate IgniteMXBean#active(boolean)?
> > > > Does this make sense? Am I missing something?
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
> > > >
> > > > > Vyacheslav.
> > > > >
> > > > > What is your suggestion for the API?
> > > > >
> > > > > Single implementation for both Ignite#active(boolean) and
> > > > > IgniteMXBean#active(boolean)
> > > > > Should public java API continue to silently clears in-memory
> caches?
> > > > >
> > > > >
> > > > > > 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <
> > > slava.koptilin@gmail.com
> > > > >
> > > > > написал(а):
> > > > > >
> > > > > > Hello Vladimir,
> > > > > >
> > > > > >> adding a new method with force flag means old methods change
> their
> > > > > > behavior:
> > > > > > I don't think that changing the behavior of public API is the
> right
> > > > way.
> > > > > > Moreover, I agree with Alex that there is no need to introduce a
> > > > > > "confirmation" flag to the java API.
> > > > > >
> > > > > > Thanks,
> > > > > > S.
> > > > > >
> > > > > > пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <
> vladsz83@gmail.com
> > >:
> > > > > >
> > > > > >> Alexey, adding a new method with force flag means old methods
> > change
> > > > > their
> > > > > >> behavior: they are considered as executed without ‘force‘ flag
> and
> > > can
> > > > > fail
> > > > > >> to prevent data loss. Ignite and IgniteMXBean are different
> > > > interfaces.
> > > > > >> Unfortunately, they have same method
> > > > > >>
> > > > > >> void  active(boolean active)
> > > > > >>
> > > > > >> When executed as IgniteMXBean it should fail if user can lose
> > data.
> > > > When
> > > > > >> executed from code via interface Ignite probably not. To solve
> > this
> > > I
> > > > > >> suggest to add ‘force’ flag for every deactivation mode:
> > > CLI/JMX/REST
> > > > > and
> > > > > >> other API.
> > > > > >>
> > > > > >> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > >>> :
> > > > > >>
> > > > > >>> Igniters,
> > > > > >>>
> > > > > >>> Do we really need the confirmation flag on the public API? I
> > > > absolutely
> > > > > >>> agree on the CLI and MXBean, but what is the reason for the
> flag
> > in
> > > > the
> > > > > >>> API? It will be specified at the compile time anyway and does
> not
> > > > > prevent
> > > > > >>> any user error.
> > > > > >>> From the implementation point of view I see no contradiction -
> we
> > > can
> > > > > add
> > > > > >>> the new method to the MXBean, but nothing forces us to add it
> to
> > > > Ignite
> > > > > >>> interface - those interfaces are not related.
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Vyacheslav,



>>> I am talking about MIXED cluster with persistent cache and *in-memory*
cache which is backed by *3-rd party persistence*.



We do not know what exactly does *3-rd party persistence*. It may store
only specific filtered data, small part of data. I think it is out of
responsibility of the persistence. Such caches might be considered as
in-memory only.



>>> That is why I do not like to expose such functionality through JMX.

But it is exposed also in CLI and REST. Just various call types. Why hide
it from JMX? Or why it is supposed to act differently? If CLI (and REST)
requires forced deactivation, why JMX would act in other way at the same
conditions?

пт, 14 февр. 2020 г. в 18:40, Вячеслав Коптилин <sl...@gmail.com>:

> Hi Vladimir,
>
> > if you have only persistent caches, no warning/confirmation is supposed
> at all.
> I am talking about MIXED cluster with persistent cache and *in-memory*
> cache which is backed by *3-rd party persistence*.
>
> > I’m afraid this won’t stop anyone from using old deprecated
> IgniteMXBean#active(boolean)
> That is why I do not like to expose such functionality through JMX.
>
> Thanks,
> S.
>
> пт, 14 февр. 2020 г. в 18:02, Vladimir Steshin <vl...@gmail.com>:
>
> >  Vyacheslav,
> >
> > >>> Let's assume that I have a mixed cluster with persistent cache and
> > in-memory cache which is backed by 3-rd party persistence. I see no
> reason
> > to throw an exception in that case at least.
> >
> >
> >
> > if you have only persistent caches, no warning/confirmation is supposed
> at
> > all.
> >
> >
> >
> > >>> Is it possible to
> > add new methods as follows: activateCluster()/deactivateCluster() and
> > deprecate IgniteMXBean#active(boolean)?
> >
> >
> >
> > I’m afraid this won’t stop anyone from using old deprecated
> > IgniteMXBean#active(boolean).
> > It is quite obvious to execute through JMX despite it is deprecated.
> >
> > пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <
> slava.koptilin@gmail.com
> > >:
> >
> > > Hello Nikolay,
> > >
> > > > Should public java API continue to silently clear in-memory caches?
> > > Let's assume that I have a mixed cluster with persistent cache and
> > > in-memory cache which is backed by 3-rd party persistence. I see no
> > reason
> > > to throw an exception in that case at least.
> > > Anyway, this fact should be clearly stated in the Javadoc and
> > documentation
> > > of course.
> > >
> > > > What is your suggestion for the API?
> > > I think we are talking about JMX methods. Am I correct? Is it possible
> to
> > > add new methods as follows: activateCluster()/deactivateCluster() and
> > > deprecate IgniteMXBean#active(boolean)?
> > > Does this make sense? Am I missing something?
> > >
> > > Thanks,
> > > S.
> > >
> > > пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
> > >
> > > > Vyacheslav.
> > > >
> > > > What is your suggestion for the API?
> > > >
> > > > Single implementation for both Ignite#active(boolean) and
> > > > IgniteMXBean#active(boolean)
> > > > Should public java API continue to silently clears in-memory caches?
> > > >
> > > >
> > > > > 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <
> > slava.koptilin@gmail.com
> > > >
> > > > написал(а):
> > > > >
> > > > > Hello Vladimir,
> > > > >
> > > > >> adding a new method with force flag means old methods change their
> > > > > behavior:
> > > > > I don't think that changing the behavior of public API is the right
> > > way.
> > > > > Moreover, I agree with Alex that there is no need to introduce a
> > > > > "confirmation" flag to the java API.
> > > > >
> > > > > Thanks,
> > > > > S.
> > > > >
> > > > > пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vladsz83@gmail.com
> >:
> > > > >
> > > > >> Alexey, adding a new method with force flag means old methods
> change
> > > > their
> > > > >> behavior: they are considered as executed without ‘force‘ flag and
> > can
> > > > fail
> > > > >> to prevent data loss. Ignite and IgniteMXBean are different
> > > interfaces.
> > > > >> Unfortunately, they have same method
> > > > >>
> > > > >> void  active(boolean active)
> > > > >>
> > > > >> When executed as IgniteMXBean it should fail if user can lose
> data.
> > > When
> > > > >> executed from code via interface Ignite probably not. To solve
> this
> > I
> > > > >> suggest to add ‘force’ flag for every deactivation mode:
> > CLI/JMX/REST
> > > > and
> > > > >> other API.
> > > > >>
> > > > >> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > >>> :
> > > > >>
> > > > >>> Igniters,
> > > > >>>
> > > > >>> Do we really need the confirmation flag on the public API? I
> > > absolutely
> > > > >>> agree on the CLI and MXBean, but what is the reason for the flag
> in
> > > the
> > > > >>> API? It will be specified at the compile time anyway and does not
> > > > prevent
> > > > >>> any user error.
> > > > >>> From the implementation point of view I see no contradiction - we
> > can
> > > > add
> > > > >>> the new method to the MXBean, but nothing forces us to add it to
> > > Ignite
> > > > >>> interface - those interfaces are not related.
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hi Vladimir,

> if you have only persistent caches, no warning/confirmation is supposed
at all.
I am talking about MIXED cluster with persistent cache and *in-memory*
cache which is backed by *3-rd party persistence*.

> I’m afraid this won’t stop anyone from using old deprecated
IgniteMXBean#active(boolean)
That is why I do not like to expose such functionality through JMX.

Thanks,
S.

пт, 14 февр. 2020 г. в 18:02, Vladimir Steshin <vl...@gmail.com>:

>  Vyacheslav,
>
> >>> Let's assume that I have a mixed cluster with persistent cache and
> in-memory cache which is backed by 3-rd party persistence. I see no reason
> to throw an exception in that case at least.
>
>
>
> if you have only persistent caches, no warning/confirmation is supposed at
> all.
>
>
>
> >>> Is it possible to
> add new methods as follows: activateCluster()/deactivateCluster() and
> deprecate IgniteMXBean#active(boolean)?
>
>
>
> I’m afraid this won’t stop anyone from using old deprecated
> IgniteMXBean#active(boolean).
> It is quite obvious to execute through JMX despite it is deprecated.
>
> пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <slava.koptilin@gmail.com
> >:
>
> > Hello Nikolay,
> >
> > > Should public java API continue to silently clear in-memory caches?
> > Let's assume that I have a mixed cluster with persistent cache and
> > in-memory cache which is backed by 3-rd party persistence. I see no
> reason
> > to throw an exception in that case at least.
> > Anyway, this fact should be clearly stated in the Javadoc and
> documentation
> > of course.
> >
> > > What is your suggestion for the API?
> > I think we are talking about JMX methods. Am I correct? Is it possible to
> > add new methods as follows: activateCluster()/deactivateCluster() and
> > deprecate IgniteMXBean#active(boolean)?
> > Does this make sense? Am I missing something?
> >
> > Thanks,
> > S.
> >
> > пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
> >
> > > Vyacheslav.
> > >
> > > What is your suggestion for the API?
> > >
> > > Single implementation for both Ignite#active(boolean) and
> > > IgniteMXBean#active(boolean)
> > > Should public java API continue to silently clears in-memory caches?
> > >
> > >
> > > > 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <
> slava.koptilin@gmail.com
> > >
> > > написал(а):
> > > >
> > > > Hello Vladimir,
> > > >
> > > >> adding a new method with force flag means old methods change their
> > > > behavior:
> > > > I don't think that changing the behavior of public API is the right
> > way.
> > > > Moreover, I agree with Alex that there is no need to introduce a
> > > > "confirmation" flag to the java API.
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
> > > >
> > > >> Alexey, adding a new method with force flag means old methods change
> > > their
> > > >> behavior: they are considered as executed without ‘force‘ flag and
> can
> > > fail
> > > >> to prevent data loss. Ignite and IgniteMXBean are different
> > interfaces.
> > > >> Unfortunately, they have same method
> > > >>
> > > >> void  active(boolean active)
> > > >>
> > > >> When executed as IgniteMXBean it should fail if user can lose data.
> > When
> > > >> executed from code via interface Ignite probably not. To solve this
> I
> > > >> suggest to add ‘force’ flag for every deactivation mode:
> CLI/JMX/REST
> > > and
> > > >> other API.
> > > >>
> > > >> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > >>> :
> > > >>
> > > >>> Igniters,
> > > >>>
> > > >>> Do we really need the confirmation flag on the public API? I
> > absolutely
> > > >>> agree on the CLI and MXBean, but what is the reason for the flag in
> > the
> > > >>> API? It will be specified at the compile time anyway and does not
> > > prevent
> > > >>> any user error.
> > > >>> From the implementation point of view I see no contradiction - we
> can
> > > add
> > > >>> the new method to the MXBean, but nothing forces us to add it to
> > Ignite
> > > >>> interface - those interfaces are not related.
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

 Vyacheslav,

>>> Let's assume that I have a mixed cluster with persistent cache and
in-memory cache which is backed by 3-rd party persistence. I see no reason
to throw an exception in that case at least.



if you have only persistent caches, no warning/confirmation is supposed at
all.



>>> Is it possible to
add new methods as follows: activateCluster()/deactivateCluster() and
deprecate IgniteMXBean#active(boolean)?



I’m afraid this won’t stop anyone from using old deprecated
IgniteMXBean#active(boolean).
It is quite obvious to execute through JMX despite it is deprecated.

пт, 14 февр. 2020 г. в 17:36, Вячеслав Коптилин <sl...@gmail.com>:

> Hello Nikolay,
>
> > Should public java API continue to silently clear in-memory caches?
> Let's assume that I have a mixed cluster with persistent cache and
> in-memory cache which is backed by 3-rd party persistence. I see no reason
> to throw an exception in that case at least.
> Anyway, this fact should be clearly stated in the Javadoc and documentation
> of course.
>
> > What is your suggestion for the API?
> I think we are talking about JMX methods. Am I correct? Is it possible to
> add new methods as follows: activateCluster()/deactivateCluster() and
> deprecate IgniteMXBean#active(boolean)?
> Does this make sense? Am I missing something?
>
> Thanks,
> S.
>
> пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
>
> > Vyacheslav.
> >
> > What is your suggestion for the API?
> >
> > Single implementation for both Ignite#active(boolean) and
> > IgniteMXBean#active(boolean)
> > Should public java API continue to silently clears in-memory caches?
> >
> >
> > > 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <slava.koptilin@gmail.com
> >
> > написал(а):
> > >
> > > Hello Vladimir,
> > >
> > >> adding a new method with force flag means old methods change their
> > > behavior:
> > > I don't think that changing the behavior of public API is the right
> way.
> > > Moreover, I agree with Alex that there is no need to introduce a
> > > "confirmation" flag to the java API.
> > >
> > > Thanks,
> > > S.
> > >
> > > пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
> > >
> > >> Alexey, adding a new method with force flag means old methods change
> > their
> > >> behavior: they are considered as executed without ‘force‘ flag and can
> > fail
> > >> to prevent data loss. Ignite and IgniteMXBean are different
> interfaces.
> > >> Unfortunately, they have same method
> > >>
> > >> void  active(boolean active)
> > >>
> > >> When executed as IgniteMXBean it should fail if user can lose data.
> When
> > >> executed from code via interface Ignite probably not. To solve this I
> > >> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST
> > and
> > >> other API.
> > >>
> > >> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > >>> :
> > >>
> > >>> Igniters,
> > >>>
> > >>> Do we really need the confirmation flag on the public API? I
> absolutely
> > >>> agree on the CLI and MXBean, but what is the reason for the flag in
> the
> > >>> API? It will be specified at the compile time anyway and does not
> > prevent
> > >>> any user error.
> > >>> From the implementation point of view I see no contradiction - we can
> > add
> > >>> the new method to the MXBean, but nothing forces us to add it to
> Ignite
> > >>> interface - those interfaces are not related.
> > >>>
> > >>
> >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Soory.

I forgot two more methods

IgniteMXBean {
    @Deprecated
    public String clusterState();

    public void clusterState(String state);
}


> 14 февр. 2020 г., в 17:50, Nikolay Izhikov <ni...@gmail.com> написал(а):
> 
> Vyacheslav.
> 
> 
> Let’s see what we got for now:
> 
> ```
> Ignite {
> 
>    @Deprecated
>    public boolean active();
> 
>    @Deprecated
>    public void active(boolean active);
> }
> 
> IgniteMXBean {
>    @Deprecated
>    public boolean active();
> 
>    public void active(boolean active); //Please, note, this method is not deprecated. It has the same implementation as Ignite#active(boolean)
> }
> 
> IgniteCluster {
>    @Deprecated
>    public boolean active();
> 
>    @Deprecated
>    public void active(boolean active);
> 
>    public ClusterState state();
> 
>    public void state(ClusterState newState) throws IgniteException;
> }
> ```
> 
> This issue with the IgniteMXBean#active(boolean) - it has the same implementation(IgniteKernal#active) as Ignite#active.
> We can’t change IgniteMXBean#active without changing Ignite#active.
> 
> Moreover, for now, we already have an API mess for cluster activation.
> If we introduce different behavior for the different interfaces(java, JMX) it will confuse users even more.
> 
>> I see no reason to throw an exception in that case at least.
> 
> Users(and even Ignite PMCs) doesn’t expect that in-memory cache will be cleared on deactivation.
> I think we should notify users about this detail in very clear fashion - require explicit confirmation for the operation.
> 
>> Anyway, this fact should be clearly stated in the Javadoc and documentation of course.
> 
> This is default requirement for the improvement.
> It will be done.
> 
> 
> 
>> 14 февр. 2020 г., в 17:36, Вячеслав Коптилин <sl...@gmail.com> написал(а):
>> 
>> Hello Nikolay,
>> 
>>> Should public java API continue to silently clear in-memory caches?
>> Let's assume that I have a mixed cluster with persistent cache and
>> in-memory cache which is backed by 3-rd party persistence. I see no reason
>> to throw an exception in that case at least.
>> Anyway, this fact should be clearly stated in the Javadoc and documentation
>> of course.
>> 
>>> What is your suggestion for the API?
>> I think we are talking about JMX methods. Am I correct? Is it possible to
>> add new methods as follows: activateCluster()/deactivateCluster() and
>> deprecate IgniteMXBean#active(boolean)?
>> Does this make sense? Am I missing something?
>> 
>> Thanks,
>> S.
>> 
>> пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
>> 
>>> Vyacheslav.
>>> 
>>> What is your suggestion for the API?
>>> 
>>> Single implementation for both Ignite#active(boolean) and
>>> IgniteMXBean#active(boolean)
>>> Should public java API continue to silently clears in-memory caches?
>>> 
>>> 
>>>> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <sl...@gmail.com>
>>> написал(а):
>>>> 
>>>> Hello Vladimir,
>>>> 
>>>>> adding a new method with force flag means old methods change their
>>>> behavior:
>>>> I don't think that changing the behavior of public API is the right way.
>>>> Moreover, I agree with Alex that there is no need to introduce a
>>>> "confirmation" flag to the java API.
>>>> 
>>>> Thanks,
>>>> S.
>>>> 
>>>> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
>>>> 
>>>>> Alexey, adding a new method with force flag means old methods change
>>> their
>>>>> behavior: they are considered as executed without ‘force‘ flag and can
>>> fail
>>>>> to prevent data loss. Ignite and IgniteMXBean are different interfaces.
>>>>> Unfortunately, they have same method
>>>>> 
>>>>> void  active(boolean active)
>>>>> 
>>>>> When executed as IgniteMXBean it should fail if user can lose data. When
>>>>> executed from code via interface Ignite probably not. To solve this I
>>>>> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST
>>> and
>>>>> other API.
>>>>> 
>>>>> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com
>>>>>> :
>>>>> 
>>>>>> Igniters,
>>>>>> 
>>>>>> Do we really need the confirmation flag on the public API? I absolutely
>>>>>> agree on the CLI and MXBean, but what is the reason for the flag in the
>>>>>> API? It will be specified at the compile time anyway and does not
>>> prevent
>>>>>> any user error.
>>>>>> From the implementation point of view I see no contradiction - we can
>>> add
>>>>>> the new method to the MXBean, but nothing forces us to add it to Ignite
>>>>>> interface - those interfaces are not related.
>>>>>> 
>>>>> 
>>> 
>>> 
> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Vyacheslav.


Let’s see what we got for now:

```
Ignite {

    @Deprecated
    public boolean active();

    @Deprecated
    public void active(boolean active);
}

IgniteMXBean {
    @Deprecated
    public boolean active();

    public void active(boolean active); //Please, note, this method is not deprecated. It has the same implementation as Ignite#active(boolean)
}

IgniteCluster {
    @Deprecated
    public boolean active();

    @Deprecated
    public void active(boolean active);

    public ClusterState state();

    public void state(ClusterState newState) throws IgniteException;
}
```

This issue with the IgniteMXBean#active(boolean) - it has the same implementation(IgniteKernal#active) as Ignite#active.
We can’t change IgniteMXBean#active without changing Ignite#active.

Moreover, for now, we already have an API mess for cluster activation.
If we introduce different behavior for the different interfaces(java, JMX) it will confuse users even more.

> I see no reason to throw an exception in that case at least.

Users(and even Ignite PMCs) doesn’t expect that in-memory cache will be cleared on deactivation.
I think we should notify users about this detail in very clear fashion - require explicit confirmation for the operation.

> Anyway, this fact should be clearly stated in the Javadoc and documentation of course.

This is default requirement for the improvement.
It will be done.



> 14 февр. 2020 г., в 17:36, Вячеслав Коптилин <sl...@gmail.com> написал(а):
> 
> Hello Nikolay,
> 
>> Should public java API continue to silently clear in-memory caches?
> Let's assume that I have a mixed cluster with persistent cache and
> in-memory cache which is backed by 3-rd party persistence. I see no reason
> to throw an exception in that case at least.
> Anyway, this fact should be clearly stated in the Javadoc and documentation
> of course.
> 
>> What is your suggestion for the API?
> I think we are talking about JMX methods. Am I correct? Is it possible to
> add new methods as follows: activateCluster()/deactivateCluster() and
> deprecate IgniteMXBean#active(boolean)?
> Does this make sense? Am I missing something?
> 
> Thanks,
> S.
> 
> пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:
> 
>> Vyacheslav.
>> 
>> What is your suggestion for the API?
>> 
>> Single implementation for both Ignite#active(boolean) and
>> IgniteMXBean#active(boolean)
>> Should public java API continue to silently clears in-memory caches?
>> 
>> 
>>> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <sl...@gmail.com>
>> написал(а):
>>> 
>>> Hello Vladimir,
>>> 
>>>> adding a new method with force flag means old methods change their
>>> behavior:
>>> I don't think that changing the behavior of public API is the right way.
>>> Moreover, I agree with Alex that there is no need to introduce a
>>> "confirmation" flag to the java API.
>>> 
>>> Thanks,
>>> S.
>>> 
>>> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
>>> 
>>>> Alexey, adding a new method with force flag means old methods change
>> their
>>>> behavior: they are considered as executed without ‘force‘ flag and can
>> fail
>>>> to prevent data loss. Ignite and IgniteMXBean are different interfaces.
>>>> Unfortunately, they have same method
>>>> 
>>>> void  active(boolean active)
>>>> 
>>>> When executed as IgniteMXBean it should fail if user can lose data. When
>>>> executed from code via interface Ignite probably not. To solve this I
>>>> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST
>> and
>>>> other API.
>>>> 
>>>> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
>> alexey.goncharuk@gmail.com
>>>>> :
>>>> 
>>>>> Igniters,
>>>>> 
>>>>> Do we really need the confirmation flag on the public API? I absolutely
>>>>> agree on the CLI and MXBean, but what is the reason for the flag in the
>>>>> API? It will be specified at the compile time anyway and does not
>> prevent
>>>>> any user error.
>>>>> From the implementation point of view I see no contradiction - we can
>> add
>>>>> the new method to the MXBean, but nothing forces us to add it to Ignite
>>>>> interface - those interfaces are not related.
>>>>> 
>>>> 
>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hello Nikolay,

> Should public java API continue to silently clear in-memory caches?
Let's assume that I have a mixed cluster with persistent cache and
in-memory cache which is backed by 3-rd party persistence. I see no reason
to throw an exception in that case at least.
Anyway, this fact should be clearly stated in the Javadoc and documentation
of course.

> What is your suggestion for the API?
I think we are talking about JMX methods. Am I correct? Is it possible to
add new methods as follows: activateCluster()/deactivateCluster() and
deprecate IgniteMXBean#active(boolean)?
Does this make sense? Am I missing something?

Thanks,
S.

пт, 14 февр. 2020 г. в 16:17, Nikolay Izhikov <ni...@apache.org>:

> Vyacheslav.
>
> What is your suggestion for the API?
>
> Single implementation for both Ignite#active(boolean) and
> IgniteMXBean#active(boolean)
> Should public java API continue to silently clears in-memory caches?
>
>
> > 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <sl...@gmail.com>
> написал(а):
> >
> > Hello Vladimir,
> >
> >> adding a new method with force flag means old methods change their
> > behavior:
> > I don't think that changing the behavior of public API is the right way.
> > Moreover, I agree with Alex that there is no need to introduce a
> > "confirmation" flag to the java API.
> >
> > Thanks,
> > S.
> >
> > пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
> >
> >> Alexey, adding a new method with force flag means old methods change
> their
> >> behavior: they are considered as executed without ‘force‘ flag and can
> fail
> >> to prevent data loss. Ignite and IgniteMXBean are different interfaces.
> >> Unfortunately, they have same method
> >>
> >> void  active(boolean active)
> >>
> >> When executed as IgniteMXBean it should fail if user can lose data. When
> >> executed from code via interface Ignite probably not. To solve this I
> >> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST
> and
> >> other API.
> >>
> >> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> >>> :
> >>
> >>> Igniters,
> >>>
> >>> Do we really need the confirmation flag on the public API? I absolutely
> >>> agree on the CLI and MXBean, but what is the reason for the flag in the
> >>> API? It will be specified at the compile time anyway and does not
> prevent
> >>> any user error.
> >>> From the implementation point of view I see no contradiction - we can
> add
> >>> the new method to the MXBean, but nothing forces us to add it to Ignite
> >>> interface - those interfaces are not related.
> >>>
> >>
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Vyacheslav.

What is your suggestion for the API?

Single implementation for both Ignite#active(boolean) and IgniteMXBean#active(boolean)
Should public java API continue to silently clears in-memory caches?


> 14 февр. 2020 г., в 15:56, Вячеслав Коптилин <sl...@gmail.com> написал(а):
> 
> Hello Vladimir,
> 
>> adding a new method with force flag means old methods change their
> behavior:
> I don't think that changing the behavior of public API is the right way.
> Moreover, I agree with Alex that there is no need to introduce a
> "confirmation" flag to the java API.
> 
> Thanks,
> S.
> 
> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
> 
>> Alexey, adding a new method with force flag means old methods change their
>> behavior: they are considered as executed without ‘force‘ flag and can fail
>> to prevent data loss. Ignite and IgniteMXBean are different interfaces.
>> Unfortunately, they have same method
>> 
>> void  active(boolean active)
>> 
>> When executed as IgniteMXBean it should fail if user can lose data. When
>> executed from code via interface Ignite probably not. To solve this I
>> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST and
>> other API.
>> 
>> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <alexey.goncharuk@gmail.com
>>> :
>> 
>>> Igniters,
>>> 
>>> Do we really need the confirmation flag on the public API? I absolutely
>>> agree on the CLI and MXBean, but what is the reason for the flag in the
>>> API? It will be specified at the compile time anyway and does not prevent
>>> any user error.
>>> From the implementation point of view I see no contradiction - we can add
>>> the new method to the MXBean, but nothing forces us to add it to Ignite
>>> interface - those interfaces are not related.
>>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Вячеслав, Hi.

Even if previous behavior is controversial? It allows to erase data
suddenly. I would suggest to slightly change API of IgniteMXBean. It would
solve the problem of single implementation IgniteKernal#active(boolean).
But changing of API is prohibited. Another solution is to extract
implementation of IgniteMXBean from IgniteKernal. I believe this approach
is too bulky.  How often Ignite#active(false) is used outside the tests?
What is so wrong to use new IgniteCluster(INACTIVE, force:true)?


To the point, the PR is here:

пт, 14 февр. 2020 г. в 15:57, Вячеслав Коптилин <sl...@gmail.com>:

> Hello Vladimir,
>
> > adding a new method with force flag means old methods change their
> behavior:
> I don't think that changing the behavior of public API is the right way.
> Moreover, I agree with Alex that there is no need to introduce a
> "confirmation" flag to the java API.
>
> Thanks,
> S.
>
> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
>
> > Alexey, adding a new method with force flag means old methods change
> their
> > behavior: they are considered as executed without ‘force‘ flag and can
> fail
> > to prevent data loss. Ignite and IgniteMXBean are different interfaces.
> > Unfortunately, they have same method
> >
> > void  active(boolean active)
> >
> > When executed as IgniteMXBean it should fail if user can lose data. When
> > executed from code via interface Ignite probably not. To solve this I
> > suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST and
> > other API.
> >
> > пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > Igniters,
> > >
> > > Do we really need the confirmation flag on the public API? I absolutely
> > > agree on the CLI and MXBean, but what is the reason for the flag in the
> > > API? It will be specified at the compile time anyway and does not
> prevent
> > > any user error.
> > > From the implementation point of view I see no contradiction - we can
> add
> > > the new method to the MXBean, but nothing forces us to add it to Ignite
> > > interface - those interfaces are not related.
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

* The pr:  https://github.com/apache/ignite/pull/7358

One can see the solution closer.

пт, 14 февр. 2020 г. в 15:57, Вячеслав Коптилин <sl...@gmail.com>:

> Hello Vladimir,
>
> > adding a new method with force flag means old methods change their
> behavior:
> I don't think that changing the behavior of public API is the right way.
> Moreover, I agree with Alex that there is no need to introduce a
> "confirmation" flag to the java API.
>
> Thanks,
> S.
>
> пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:
>
> > Alexey, adding a new method with force flag means old methods change
> their
> > behavior: they are considered as executed without ‘force‘ flag and can
> fail
> > to prevent data loss. Ignite and IgniteMXBean are different interfaces.
> > Unfortunately, they have same method
> >
> > void  active(boolean active)
> >
> > When executed as IgniteMXBean it should fail if user can lose data. When
> > executed from code via interface Ignite probably not. To solve this I
> > suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST and
> > other API.
> >
> > пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > Igniters,
> > >
> > > Do we really need the confirmation flag on the public API? I absolutely
> > > agree on the CLI and MXBean, but what is the reason for the flag in the
> > > API? It will be specified at the compile time anyway and does not
> prevent
> > > any user error.
> > > From the implementation point of view I see no contradiction - we can
> add
> > > the new method to the MXBean, but nothing forces us to add it to Ignite
> > > interface - those interfaces are not related.
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Вячеслав Коптилин <sl...@gmail.com>]

Hello Vladimir,

> adding a new method with force flag means old methods change their
behavior:
I don't think that changing the behavior of public API is the right way.
Moreover, I agree with Alex that there is no need to introduce a
"confirmation" flag to the java API.

Thanks,
S.

пт, 14 февр. 2020 г. в 15:38, Vladimir Steshin <vl...@gmail.com>:

> Alexey, adding a new method with force flag means old methods change their
> behavior: they are considered as executed without ‘force‘ flag and can fail
> to prevent data loss. Ignite and IgniteMXBean are different interfaces.
> Unfortunately, they have same method
>
> void  active(boolean active)
>
> When executed as IgniteMXBean it should fail if user can lose data. When
> executed from code via interface Ignite probably not. To solve this I
> suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST and
> other API.
>
> пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > Igniters,
> >
> > Do we really need the confirmation flag on the public API? I absolutely
> > agree on the CLI and MXBean, but what is the reason for the flag in the
> > API? It will be specified at the compile time anyway and does not prevent
> > any user error.
> > From the implementation point of view I see no contradiction - we can add
> > the new method to the MXBean, but nothing forces us to add it to Ignite
> > interface - those interfaces are not related.
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Alexey, adding a new method with force flag means old methods change their
behavior: they are considered as executed without ‘force‘ flag and can fail
to prevent data loss. Ignite and IgniteMXBean are different interfaces.
Unfortunately, they have same method

void  active(boolean active)

When executed as IgniteMXBean it should fail if user can lose data. When
executed from code via interface Ignite probably not. To solve this I
suggest to add ‘force’ flag for every deactivation mode: CLI/JMX/REST and
other API.

пт, 14 февр. 2020 г. в 15:20, Alexey Goncharuk <al...@gmail.com>:

> Igniters,
>
> Do we really need the confirmation flag on the public API? I absolutely
> agree on the CLI and MXBean, but what is the reason for the flag in the
> API? It will be specified at the compile time anyway and does not prevent
> any user error.
> From the implementation point of view I see no contradiction - we can add
> the new method to the MXBean, but nothing forces us to add it to Ignite
> interface - those interfaces are not related.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

Igniters,

Do we really need the confirmation flag on the public API? I absolutely
agree on the CLI and MXBean, but what is the reason for the flag in the
API? It will be specified at the compile time anyway and does not prevent
any user error.
From the implementation point of view I see no contradiction - we can add
the new method to the MXBean, but nothing forces us to add it to Ignite
interface - those interfaces are not related.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikita Amelchev <ns...@gmail.com>]

Hi, guys.

I suggest mark as deprecated the IgniteMXBean#active(boolean) method.
The Ignite#active(boolean),  IgniteMXBean#active(),
IgniteCluster#active(boolean) methods already deprecated and told to
use the #state(ClusterState)} instead.

Should the old IgniteMXBean#state(ClusterState) method throw a
not-safe deactivation exception? Or it should just be marked as
deprecated?

пт, 14 февр. 2020 г. в 12:18, Nikolay Izhikov <ni...@apache.org>:
>
> Hello, Vladimir.
>
> I think we should do the following:
>
> * Update Ignite documentation and write down the fact that in-memory cache cleared on deactivation.
> * Disallow, by default, deactivation of the cluster that has in-memory cache with proper error message
>         «Your cluster has in-memory cache configured. During deactivation all data from these caches will be cleared!»
> * Add «—force» flag to deactivate command so administrator can force deactivation for cluster that has both - persistent and in-memory caches configured.
>
> Changes in the source code should include:
>
> 1. JMX beans - IgniteMXBean.
> 2. Java API - Ignite#active and Ignite#cluster#state - because active method have the same implementation as `IgniteMBBean#active`
> 3. CLI utility - control.sh
>
> Igniters, what do you think?
>
> > 13 февр. 2020 г., в 14:19, Vladimir Steshin <vl...@gmail.com> написал(а):
> >
> > Hello, everyone.
> >
> >
> >
> > I’d like to propose to make behavior of cluster deactivation the same
> > independently on usage type: in code or manually with control.sh or JMX.
> > Let’s put it in one place like IgniteCluster#state(ClusterState newState,
> > boolean force) and require forced call for in-memory data.
> >
> > While I was doing the ticket I realized that the suggested previously
> > solution cannot be complete. To prevent data loss via JMX I would need to
> > stop executing IgniteMXBean#active(false). But this causes stopping of
> > Ignite#active(false) too. The problem relies in single implementation for
> > both interfaces in IgniteKernal#active(boolean). It has to act differently
> > depending on where it is called from: JMX/CLI or code.
> >
> >
> >
> >            Any thoughts?
> >
> > чт, 30 янв. 2020 г. в 13:08, Alexey Goncharuk <al...@gmail.com>:
> >
> >> I agree on CLI and JMX because those interfaces can be used by a system
> >> administrator and can be invoked by mistake.
> >>
> >> As for the Java API, personally, I find it strange to add 'force' or
> >> 'confirm' flags to it because it is very unlikely that such an invocation
> >> is done by mistake. Such mistakes are caught during the testing phase and
> >> developers will end up hard-coding 'true' as a flag anyways.
> >>
>


-- 
Best wishes,
Amelchev Nikita

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Vladimir.

I think we should do the following:

* Update Ignite documentation and write down the fact that in-memory cache cleared on deactivation.
* Disallow, by default, deactivation of the cluster that has in-memory cache with proper error message
	«Your cluster has in-memory cache configured. During deactivation all data from these caches will be cleared!»
* Add «—force» flag to deactivate command so administrator can force deactivation for cluster that has both - persistent and in-memory caches configured.

Changes in the source code should include:

1. JMX beans - IgniteMXBean.
2. Java API - Ignite#active and Ignite#cluster#state - because active method have the same implementation as `IgniteMBBean#active`
3. CLI utility - control.sh

Igniters, what do you think?

> 13 февр. 2020 г., в 14:19, Vladimir Steshin <vl...@gmail.com> написал(а):
> 
> Hello, everyone.
> 
> 
> 
> I’d like to propose to make behavior of cluster deactivation the same
> independently on usage type: in code or manually with control.sh or JMX.
> Let’s put it in one place like IgniteCluster#state(ClusterState newState,
> boolean force) and require forced call for in-memory data.
> 
> While I was doing the ticket I realized that the suggested previously
> solution cannot be complete. To prevent data loss via JMX I would need to
> stop executing IgniteMXBean#active(false). But this causes stopping of
> Ignite#active(false) too. The problem relies in single implementation for
> both interfaces in IgniteKernal#active(boolean). It has to act differently
> depending on where it is called from: JMX/CLI or code.
> 
> 
> 
>            Any thoughts?
> 
> чт, 30 янв. 2020 г. в 13:08, Alexey Goncharuk <al...@gmail.com>:
> 
>> I agree on CLI and JMX because those interfaces can be used by a system
>> administrator and can be invoked by mistake.
>> 
>> As for the Java API, personally, I find it strange to add 'force' or
>> 'confirm' flags to it because it is very unlikely that such an invocation
>> is done by mistake. Such mistakes are caught during the testing phase and
>> developers will end up hard-coding 'true' as a flag anyways.
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hello, everyone.



I’d like to propose to make behavior of cluster deactivation the same
independently on usage type: in code or manually with control.sh or JMX.
Let’s put it in one place like IgniteCluster#state(ClusterState newState,
boolean force) and require forced call for in-memory data.

While I was doing the ticket I realized that the suggested previously
solution cannot be complete. To prevent data loss via JMX I would need to
stop executing IgniteMXBean#active(false). But this causes stopping of
Ignite#active(false) too. The problem relies in single implementation for
both interfaces in IgniteKernal#active(boolean). It has to act differently
depending on where it is called from: JMX/CLI or code.



            Any thoughts?

чт, 30 янв. 2020 г. в 13:08, Alexey Goncharuk <al...@gmail.com>:

> I agree on CLI and JMX because those interfaces can be used by a system
> administrator and can be invoked by mistake.
>
> As for the Java API, personally, I find it strange to add 'force' or
> 'confirm' flags to it because it is very unlikely that such an invocation
> is done by mistake. Such mistakes are caught during the testing phase and
> developers will end up hard-coding 'true' as a flag anyways.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Go for it, Vladimir!

> 31 янв. 2020 г., в 16:15, Vladimir Steshin <vl...@gmail.com> написал(а):
> 
> Hi folks. I created a ticket for this (IGNITE-12614):
> 
> 
> 
> ----
> 
> Currently, anyone is able to silently deactivate cluster with command line
> utility (control.sh). Probably with JMX too. Same for the API: Java API –
> Ignite.cluster.state(). That would lead to data loss when the persistence
> is off. In-memory data is erased during deactivation. Such behavior can be
> considered as unexpected to user.
> 
> Suggestions for the CLI/JMX:
> 
> 1) disallow silent deactivation of cluster keeping caches. Show a warning
> like “Your cluster has in-memory cache configured. During deactivation all
> data from these caches will be cleared!”
> 
> 2) Add param ‘--force’ which skips the warning message.
> 
> -----
> 
> Going to fix the problem if there are no disagreement
> <https://www.linguee.com/english-russian/translation/disagreement.html>s
> 
> пт, 31 янв. 2020 г. в 13:17, Ivan Pavlukhin <vo...@gmail.com>:
> 
>> For me it looks like some coincidence effect. I understand that we get
>> such behavior because deactivation works the same way as for
>> persistent caches. Was cluster activation/deactivation designed and
>> described for in-memory caches? Current behavior sounds for me a as
>> big risk. I expect a lot of upset users unexpectedly purged all their
>> data.
>> 
>> пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <alexey.goncharuk@gmail.com
>>> :
>>> 
>>> Because originally the sole purpose of deactivation was resource
>>> deallocation.
>>> 
>>> чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
>>> 
>>>> Such a revelation for me that data is purged from RAM if someone
>>>> deactivates the cluster. Alex, do you remember why we decided to
>> implement
>>>> it this way initially?
>>>> 
>>>> -
>>>> Denis
>>>> 
>>>> 
>>>> On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
>>>> alexey.goncharuk@gmail.com>
>>>> wrote:
>>>> 
>>>>> I agree on CLI and JMX because those interfaces can be used by a
>> system
>>>>> administrator and can be invoked by mistake.
>>>>> 
>>>>> As for the Java API, personally, I find it strange to add 'force' or
>>>>> 'confirm' flags to it because it is very unlikely that such an
>> invocation
>>>>> is done by mistake. Such mistakes are caught during the testing
>> phase and
>>>>> developers will end up hard-coding 'true' as a flag anyways.
>>>>> 
>>>> 
>> 
>> 
>> 
>> --
>> Best regards,
>> Ivan Pavlukhin
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hi folks. I created a ticket for this (IGNITE-12614):



----

Currently, anyone is able to silently deactivate cluster with command line
utility (control.sh). Probably with JMX too. Same for the API: Java API –
Ignite.cluster.state(). That would lead to data loss when the persistence
is off. In-memory data is erased during deactivation. Such behavior can be
considered as unexpected to user.

Suggestions for the CLI/JMX:

1) disallow silent deactivation of cluster keeping caches. Show a warning
like “Your cluster has in-memory cache configured. During deactivation all
data from these caches will be cleared!”

2) Add param ‘--force’ which skips the warning message.

-----

Going to fix the problem if there are no disagreement
<https://www.linguee.com/english-russian/translation/disagreement.html>s

пт, 31 янв. 2020 г. в 13:17, Ivan Pavlukhin <vo...@gmail.com>:

> For me it looks like some coincidence effect. I understand that we get
> such behavior because deactivation works the same way as for
> persistent caches. Was cluster activation/deactivation designed and
> described for in-memory caches? Current behavior sounds for me a as
> big risk. I expect a lot of upset users unexpectedly purged all their
> data.
>
> пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > Because originally the sole purpose of deactivation was resource
> > deallocation.
> >
> > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> >
> > > Such a revelation for me that data is purged from RAM if someone
> > > deactivates the cluster. Alex, do you remember why we decided to
> implement
> > > it this way initially?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com>
> > > wrote:
> > >
> > > > I agree on CLI and JMX because those interfaces can be used by a
> system
> > > > administrator and can be invoked by mistake.
> > > >
> > > > As for the Java API, personally, I find it strange to add 'force' or
> > > > 'confirm' flags to it because it is very unlikely that such an
> invocation
> > > > is done by mistake. Such mistakes are caught during the testing
> phase and
> > > > developers will end up hard-coding 'true' as a flag anyways.
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hi, folks.

Another question has raised. Can I remove / rename / bring substitution for
IgniteMXBean#active(boolean active) ? Or at least to deprecate?

The problem is that it interferes with Ignite#active(boolean active). Same
signature for different interfaces. And only one implementation:
IgniteKernal#active(boolean). We decided not to touch internal calls to
cluster deactivation. We warn user only at manual deactivation via CLI and
JMX. In the code and tests one can find many usages of
Ignite#active(boolean active). I can’t, for example, throw an exception
from it. Several tests will fail. Including C++ and C# ones. But this also
means IgniteMXBean#active(boolean active) won’t warn user of the data loss
too. Adding new method to the JMX bean won’t solve the problem. Old
active(boolean) comes first, clearly seen and is obvious to use.

What is the best practice for cases like this?

пт, 7 февр. 2020 г. в 11:23, Nikolay Izhikov <ni...@apache.org>:

> > To prevent unexpected data loss on
> deactivation CLI and JMX should require 'force' flag
>
>  Vladimir, please, go for it.
>
>
> пт, 7 февр. 2020 г., 11:19 Vladimir Steshin <vl...@gmail.com>:
>
> > Ok. Then we are at the beginning. To prevent unexpected data loss on
> > deactivation CLI and JMX should require 'force' flag. If there are no
> extra
> > proposals I'm going to finish the pull request.
> >
> > чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com
> > >:
> >
> > > Vladimir,
> > >
> > > IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> > > clearing.
> > > It allows allocated memory reuse on re-activation to avoid OS specific
> > > issues if allocated heap is rather large.
> > >
> > > You are right to create separate ticket for implementing required
> > behavior.
> > >
> > > чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
> > >
> > > > Or, is flag [1] is actual only for persistence mode? Related ticket
> [2]
> > > is
> > > > completely about disabled persistence.
> > > > If previous assumption is right, I think, we can involve flag [1] in
> > > ticket
> > > > [2].
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> > > >
> > > > > Denis, Alexei,
> > > > >
> > > > > Regarding usage of flag
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > > [1]
> > > > >
> > > > > When enabled, I think the following test should work. But it fails.
> > > > >
> > > > > //----------------------------------------------------------------
> > > > >     @Test
> > > > >     public void testDataPresent() throws Exception {
> > > > >         System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE,
> > "true");
> > > > >
> > > > >         IgniteEx i = startGrid(0);
> > > > >
> > > > >         assertFalse(
> > > > >
> > > >
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > > > >             .isPersistenceEnabled() );
> > > > >
> > > > >         String name = "non-persistent-cache";
> > > > >
> > > > >         i.createCache(name).put(1L, 1L);
> > > > >
> > > > >         assertEquals(1L, i.cache(name).get(1L));
> > > > >
> > > > >         i.cluster().state(ClusterState.INACTIVE);
> > > > >         i.cluster().state(ClusterState.ACTIVE);
> > > > >
> > > > >         assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> > > here!
> > > > >     }
> > > > > //----------------------------------------------------------------
> > > > >
> > > > >
> > > > > Several notes:
> > > > >
> > > > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > > > >                   is true
> > > > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > > > >      is called with shutdown == false
> > > > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > > > >                      is called with deallocate == false
> > > > >
> > > > > But the cache from the test still has zero size after reactivation.
> > > > >
> > > > > Is flag [1] disabled by default because it is not implemented /
> > doesn't
> > > > > work? Do we need to skip it in current ticket and rise new one?
> > > > >
> > > > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > > > >
> > > > >> I believe there might be a consistency-related reason why this
> flag
> > > was
> > > > >> disabled by default for caches that store data in Ignite native
> > > > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some
> > light
> > > on
> > > > >> this.
> > > > >>
> > > > >> As for the memory-only caches or caches backed up by a CacheStore
> > such
> > > > as
> > > > >> an RDBMS, enabling of the flag should be harmless. Once we do
> this,
> > > > we'll
> > > > >> eliminate the need to load the data back into the cluster which
> can
> > > be a
> > > > >> time-consuming operation depending on the data volume.
> > > > >>
> > > > >>
> > > > >> -
> > > > >> Denis
> > > > >>
> > > > >>
> > > > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <
> > vladsz83@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> > > There
> > > > >> > should be a reason for that. Any data consistency issues when
> node
> > > > gets
> > > > >> > activated anew? We may use both solutions because the flag can
> be
> > > > >> switched
> > > > >> > off again.
> > > > >> >
> > > > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > > > >> >
> > > > >> > > Hi Vladimir,
> > > > >> > >
> > > > >> > > Yes, I'm suggesting us to enable this flag by
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > default instead of introducing --force flag and showing any
> > > > warnings.
> > > > >> > >
> > > > >> > > -
> > > > >> > > Denis
> > > > >> > >
> > > > >> > >
> > > > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> > > vladsz83@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hello all.
> > > > >> > > >
> > > > >> > > > Denis, which changes exactly? In current implementation of
> > > ticket
> > > > >> [2]
> > > > >> > > flag
> > > > >> > > > [1] is checked before requiring --force flag and showing any
> > > > >> warnings.
> > > > >> > Do
> > > > >> > > > we need to set reuse-memory-on-deactivate to true by
> default?
> > > > >> > > >
> > > > >> > > > [1]
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > >
> > > > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dmagda@apache.org
> >:
> > > > >> > > >
> > > > >> > > > > That's the best solution for this scenario. Should we
> > readjust
> > > > the
> > > > >> > > > already
> > > > >> > > > > created ticket [1] suggesting to implement the changes of
> > Alex
> > > > >> > > Scherbakov
> > > > >> > > > > instead?
> > > > >> > > > >
> > > > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >> > > > >
> > > > >> > > > > -
> > > > >> > > > > Denis
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Folks,
> > > > >> > > > > >
> > > > >> > > > > > For a long time we have a flag [1]
> > > > >> > > > > >
> > > > >> > > > > > It does almost what we want here.
> > > > >> > > > > >
> > > > >> > > > > > I suggest to make this behavior default and rework it to
> > > keep
> > > > >> data
> > > > >> > in
> > > > >> > > > > > memory as well (we already have special "recovery" mode
> > for
> > > > >> > caches).
> > > > >> > > > > >
> > > > >> > > > > > [1]
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > >> > > > > alexey.goncharuk@gmail.com
> > > > >> > > > > > >:
> > > > >> > > > > >
> > > > >> > > > > > > I do not mind making this change if we explicitly and
> > > > clearly
> > > > >> > > define
> > > > >> > > > > what
> > > > >> > > > > > > 'new inactive state' means. What should happen if a
> > > > partition
> > > > >> is
> > > > >> > > lost
> > > > >> > > > > in
> > > > >> > > > > > > inactive state? What if such node joins back the
> cluster
> > > > >> after?
> > > > >> > > Etc.
> > > > >> > > > > > >
> > > > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> > > dmagda@apache.org
> > > > >:
> > > > >> > > > > > >
> > > > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > > >> > > activation/deactivation
> > > > >> > > > > was
> > > > >> > > > > > > > created for the baseline topology designed for cases
> > > with
> > > > >> > native
> > > > >> > > > > > > > persistence. My thinking was that the mechanism
> > itended
> > > to
> > > > >> > > prevent
> > > > >> > > > > data
> > > > >> > > > > > > > inconsistencies while nodes with data on the disk
> will
> > > be
> > > > in
> > > > >> > the
> > > > >> > > > > > process
> > > > >> > > > > > > of
> > > > >> > > > > > > > joining the cluster.
> > > > >> > > > > > > >
> > > > >> > > > > > > > Artem, could you please update the docs bringing
> this
> > to
> > > > the
> > > > >> > > > > attention
> > > > >> > > > > > of
> > > > >> > > > > > > > the user community?
> > > > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > >> > > > > > > >
> > > > >> > > > > > > > AG, what if we don't purge data from memory at least
> > for
> > > > the
> > > > >> > > caches
> > > > >> > > > > not
> > > > >> > > > > > > > backed by the native persistence? Is this a big
> deal?
> > We
> > > > can
> > > > >> > > > > certainly
> > > > >> > > > > > > put
> > > > >> > > > > > > > this off by my guts feel we'll return to this
> question
> > > > >> sooner
> > > > >> > or
> > > > >> > > > > later.
> > > > >> > > > > > > >
> > > > >> > > > > > > > -
> > > > >> > > > > > > > Denis
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > > >> > > > vololo100@gmail.com>
> > > > >> > > > > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > For me it looks like some coincidence effect. I
> > > > understand
> > > > >> > that
> > > > >> > > > we
> > > > >> > > > > > get
> > > > >> > > > > > > > > such behavior because deactivation works the same
> > way
> > > as
> > > > >> for
> > > > >> > > > > > > > > persistent caches. Was cluster
> > activation/deactivation
> > > > >> > designed
> > > > >> > > > and
> > > > >> > > > > > > > > described for in-memory caches? Current behavior
> > > sounds
> > > > >> for
> > > > >> > me
> > > > >> > > a
> > > > >> > > > as
> > > > >> > > > > > > > > big risk. I expect a lot of upset users
> unexpectedly
> > > > >> purged
> > > > >> > all
> > > > >> > > > > their
> > > > >> > > > > > > > > data.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > >> > > > > > > > alexey.goncharuk@gmail.com
> > > > >> > > > > > > > > >:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Because originally the sole purpose of
> > deactivation
> > > > was
> > > > >> > > > resource
> > > > >> > > > > > > > > > deallocation.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > > > >> > dmagda@apache.org
> > > > >> > > >:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > > Such a revelation for me that data is purged
> > from
> > > > RAM
> > > > >> if
> > > > >> > > > > someone
> > > > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember
> > why
> > > > we
> > > > >> > > decided
> > > > >> > > > > to
> > > > >> > > > > > > > > implement
> > > > >> > > > > > > > > > > it this way initially?
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > -
> > > > >> > > > > > > > > > > Denis
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey
> > Goncharuk <
> > > > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > >> > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > > I agree on CLI and JMX because those
> > interfaces
> > > > can
> > > > >> be
> > > > >> > > used
> > > > >> > > > > by
> > > > >> > > > > > a
> > > > >> > > > > > > > > system
> > > > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > As for the Java API, personally, I find it
> > > strange
> > > > >> to
> > > > >> > add
> > > > >> > > > > > 'force'
> > > > >> > > > > > > > or
> > > > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> > > unlikely
> > > > >> that
> > > > >> > > such
> > > > >> > > > > an
> > > > >> > > > > > > > > invocation
> > > > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > > > during
> > > > >> the
> > > > >> > > > > testing
> > > > >> > > > > > > > > phase and
> > > > >> > > > > > > > > > > > developers will end up hard-coding 'true'
> as a
> > > > flag
> > > > >> > > > anyways.
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > --
> > > > >> > > > > > > > > Best regards,
> > > > >> > > > > > > > > Ivan Pavlukhin
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > --
> > > > >> > > > > >
> > > > >> > > > > > Best regards,
> > > > >> > > > > > Alexei Scherbakov
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

> To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag

 Vladimir, please, go for it.


пт, 7 февр. 2020 г., 11:19 Vladimir Steshin <vl...@gmail.com>:

> Ok. Then we are at the beginning. To prevent unexpected data loss on
> deactivation CLI and JMX should require 'force' flag. If there are no extra
> proposals I'm going to finish the pull request.
>
> чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com
> >:
>
> > Vladimir,
> >
> > IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> > clearing.
> > It allows allocated memory reuse on re-activation to avoid OS specific
> > issues if allocated heap is rather large.
> >
> > You are right to create separate ticket for implementing required
> behavior.
> >
> > чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
> >
> > > Or, is flag [1] is actual only for persistence mode? Related ticket [2]
> > is
> > > completely about disabled persistence.
> > > If previous assumption is right, I think, we can involve flag [1] in
> > ticket
> > > [2].
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> > >
> > > > Denis, Alexei,
> > > >
> > > > Regarding usage of flag
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > [1]
> > > >
> > > > When enabled, I think the following test should work. But it fails.
> > > >
> > > > //----------------------------------------------------------------
> > > >     @Test
> > > >     public void testDataPresent() throws Exception {
> > > >         System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE,
> "true");
> > > >
> > > >         IgniteEx i = startGrid(0);
> > > >
> > > >         assertFalse(
> > > >
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > > >             .isPersistenceEnabled() );
> > > >
> > > >         String name = "non-persistent-cache";
> > > >
> > > >         i.createCache(name).put(1L, 1L);
> > > >
> > > >         assertEquals(1L, i.cache(name).get(1L));
> > > >
> > > >         i.cluster().state(ClusterState.INACTIVE);
> > > >         i.cluster().state(ClusterState.ACTIVE);
> > > >
> > > >         assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> > here!
> > > >     }
> > > > //----------------------------------------------------------------
> > > >
> > > >
> > > > Several notes:
> > > >
> > > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > > >                   is true
> > > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > > >      is called with shutdown == false
> > > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > > >                      is called with deallocate == false
> > > >
> > > > But the cache from the test still has zero size after reactivation.
> > > >
> > > > Is flag [1] disabled by default because it is not implemented /
> doesn't
> > > > work? Do we need to skip it in current ticket and rise new one?
> > > >
> > > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > > >
> > > >> I believe there might be a consistency-related reason why this flag
> > was
> > > >> disabled by default for caches that store data in Ignite native
> > > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some
> light
> > on
> > > >> this.
> > > >>
> > > >> As for the memory-only caches or caches backed up by a CacheStore
> such
> > > as
> > > >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> > > we'll
> > > >> eliminate the need to load the data back into the cluster which can
> > be a
> > > >> time-consuming operation depending on the data volume.
> > > >>
> > > >>
> > > >> -
> > > >> Denis
> > > >>
> > > >>
> > > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <
> vladsz83@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> > There
> > > >> > should be a reason for that. Any data consistency issues when node
> > > gets
> > > >> > activated anew? We may use both solutions because the flag can be
> > > >> switched
> > > >> > off again.
> > > >> >
> > > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > > >> >
> > > >> > > Hi Vladimir,
> > > >> > >
> > > >> > > Yes, I'm suggesting us to enable this flag by
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > default instead of introducing --force flag and showing any
> > > warnings.
> > > >> > >
> > > >> > > -
> > > >> > > Denis
> > > >> > >
> > > >> > >
> > > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> > vladsz83@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hello all.
> > > >> > > >
> > > >> > > > Denis, which changes exactly? In current implementation of
> > ticket
> > > >> [2]
> > > >> > > flag
> > > >> > > > [1] is checked before requiring --force flag and showing any
> > > >> warnings.
> > > >> > Do
> > > >> > > > we need to set reuse-memory-on-deactivate to true by default?
> > > >> > > >
> > > >> > > > [1]
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > >
> > > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >> > > >
> > > >> > > >
> > > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > > >> > > >
> > > >> > > > > That's the best solution for this scenario. Should we
> readjust
> > > the
> > > >> > > > already
> > > >> > > > > created ticket [1] suggesting to implement the changes of
> Alex
> > > >> > > Scherbakov
> > > >> > > > > instead?
> > > >> > > > >
> > > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >> > > > >
> > > >> > > > > -
> > > >> > > > > Denis
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > >> > > > >
> > > >> > > > > > Folks,
> > > >> > > > > >
> > > >> > > > > > For a long time we have a flag [1]
> > > >> > > > > >
> > > >> > > > > > It does almost what we want here.
> > > >> > > > > >
> > > >> > > > > > I suggest to make this behavior default and rework it to
> > keep
> > > >> data
> > > >> > in
> > > >> > > > > > memory as well (we already have special "recovery" mode
> for
> > > >> > caches).
> > > >> > > > > >
> > > >> > > > > > [1]
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > >> > > > > alexey.goncharuk@gmail.com
> > > >> > > > > > >:
> > > >> > > > > >
> > > >> > > > > > > I do not mind making this change if we explicitly and
> > > clearly
> > > >> > > define
> > > >> > > > > what
> > > >> > > > > > > 'new inactive state' means. What should happen if a
> > > partition
> > > >> is
> > > >> > > lost
> > > >> > > > > in
> > > >> > > > > > > inactive state? What if such node joins back the cluster
> > > >> after?
> > > >> > > Etc.
> > > >> > > > > > >
> > > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> > dmagda@apache.org
> > > >:
> > > >> > > > > > >
> > > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > >> > > activation/deactivation
> > > >> > > > > was
> > > >> > > > > > > > created for the baseline topology designed for cases
> > with
> > > >> > native
> > > >> > > > > > > > persistence. My thinking was that the mechanism
> itended
> > to
> > > >> > > prevent
> > > >> > > > > data
> > > >> > > > > > > > inconsistencies while nodes with data on the disk will
> > be
> > > in
> > > >> > the
> > > >> > > > > > process
> > > >> > > > > > > of
> > > >> > > > > > > > joining the cluster.
> > > >> > > > > > > >
> > > >> > > > > > > > Artem, could you please update the docs bringing this
> to
> > > the
> > > >> > > > > attention
> > > >> > > > > > of
> > > >> > > > > > > > the user community?
> > > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > >> > > > > > > >
> > > >> > > > > > > > AG, what if we don't purge data from memory at least
> for
> > > the
> > > >> > > caches
> > > >> > > > > not
> > > >> > > > > > > > backed by the native persistence? Is this a big deal?
> We
> > > can
> > > >> > > > > certainly
> > > >> > > > > > > put
> > > >> > > > > > > > this off by my guts feel we'll return to this question
> > > >> sooner
> > > >> > or
> > > >> > > > > later.
> > > >> > > > > > > >
> > > >> > > > > > > > -
> > > >> > > > > > > > Denis
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > >> > > > vololo100@gmail.com>
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > For me it looks like some coincidence effect. I
> > > understand
> > > >> > that
> > > >> > > > we
> > > >> > > > > > get
> > > >> > > > > > > > > such behavior because deactivation works the same
> way
> > as
> > > >> for
> > > >> > > > > > > > > persistent caches. Was cluster
> activation/deactivation
> > > >> > designed
> > > >> > > > and
> > > >> > > > > > > > > described for in-memory caches? Current behavior
> > sounds
> > > >> for
> > > >> > me
> > > >> > > a
> > > >> > > > as
> > > >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> > > >> purged
> > > >> > all
> > > >> > > > > their
> > > >> > > > > > > > > data.
> > > >> > > > > > > > >
> > > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > >> > > > > > > > alexey.goncharuk@gmail.com
> > > >> > > > > > > > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Because originally the sole purpose of
> deactivation
> > > was
> > > >> > > > resource
> > > >> > > > > > > > > > deallocation.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > > >> > dmagda@apache.org
> > > >> > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > > Such a revelation for me that data is purged
> from
> > > RAM
> > > >> if
> > > >> > > > > someone
> > > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember
> why
> > > we
> > > >> > > decided
> > > >> > > > > to
> > > >> > > > > > > > > implement
> > > >> > > > > > > > > > > it this way initially?
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > -
> > > >> > > > > > > > > > > Denis
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey
> Goncharuk <
> > > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > >> > > > > > > > > > > wrote:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > > I agree on CLI and JMX because those
> interfaces
> > > can
> > > >> be
> > > >> > > used
> > > >> > > > > by
> > > >> > > > > > a
> > > >> > > > > > > > > system
> > > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > As for the Java API, personally, I find it
> > strange
> > > >> to
> > > >> > add
> > > >> > > > > > 'force'
> > > >> > > > > > > > or
> > > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> > unlikely
> > > >> that
> > > >> > > such
> > > >> > > > > an
> > > >> > > > > > > > > invocation
> > > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > > during
> > > >> the
> > > >> > > > > testing
> > > >> > > > > > > > > phase and
> > > >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> > > flag
> > > >> > > > anyways.
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > --
> > > >> > > > > > > > > Best regards,
> > > >> > > > > > > > > Ivan Pavlukhin
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > --
> > > >> > > > > >
> > > >> > > > > > Best regards,
> > > >> > > > > > Alexei Scherbakov
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Ok. Then we are at the beginning. To prevent unexpected data loss on
deactivation CLI and JMX should require 'force' flag. If there are no extra
proposals I'm going to finish the pull request.

чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:

> Vladimir,
>
> IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
> clearing.
> It allows allocated memory reuse on re-activation to avoid OS specific
> issues if allocated heap is rather large.
>
> You are right to create separate ticket for implementing required behavior.
>
> чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:
>
> > Or, is flag [1] is actual only for persistence mode? Related ticket [2]
> is
> > completely about disabled persistence.
> > If previous assumption is right, I think, we can involve flag [1] in
> ticket
> > [2].
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
> >
> > > Denis, Alexei,
> > >
> > > Regarding usage of flag
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > [1]
> > >
> > > When enabled, I think the following test should work. But it fails.
> > >
> > > //----------------------------------------------------------------
> > >     @Test
> > >     public void testDataPresent() throws Exception {
> > >         System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
> > >
> > >         IgniteEx i = startGrid(0);
> > >
> > >         assertFalse(
> > >
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> > >             .isPersistenceEnabled() );
> > >
> > >         String name = "non-persistent-cache";
> > >
> > >         i.createCache(name).put(1L, 1L);
> > >
> > >         assertEquals(1L, i.cache(name).get(1L));
> > >
> > >         i.cluster().state(ClusterState.INACTIVE);
> > >         i.cluster().state(ClusterState.ACTIVE);
> > >
> > >         assertEquals(1L, i.cache(name).get(1L)); //Assertion error
> here!
> > >     }
> > > //----------------------------------------------------------------
> > >
> > >
> > > Several notes:
> > >
> > > - IgniteCacheDatabaseSharedManager#reuseMemory
> > >                   is true
> > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> > >      is called with shutdown == false
> > > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> > >                      is called with deallocate == false
> > >
> > > But the cache from the test still has zero size after reactivation.
> > >
> > > Is flag [1] disabled by default because it is not implemented / doesn't
> > > work? Do we need to skip it in current ticket and rise new one?
> > >
> > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> > >
> > >> I believe there might be a consistency-related reason why this flag
> was
> > >> disabled by default for caches that store data in Ignite native
> > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light
> on
> > >> this.
> > >>
> > >> As for the memory-only caches or caches backed up by a CacheStore such
> > as
> > >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> > we'll
> > >> eliminate the need to load the data back into the cluster which can
> be a
> > >> time-consuming operation depending on the data volume.
> > >>
> > >>
> > >> -
> > >> Denis
> > >>
> > >>
> > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> > >> wrote:
> > >>
> > >> > Denis, but why reuse-data-on-deactivate was disabled by default?
> There
> > >> > should be a reason for that. Any data consistency issues when node
> > gets
> > >> > activated anew? We may use both solutions because the flag can be
> > >> switched
> > >> > off again.
> > >> >
> > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> > >> >
> > >> > > Hi Vladimir,
> > >> > >
> > >> > > Yes, I'm suggesting us to enable this flag by
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > default instead of introducing --force flag and showing any
> > warnings.
> > >> > >
> > >> > > -
> > >> > > Denis
> > >> > >
> > >> > >
> > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <
> vladsz83@gmail.com
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Hello all.
> > >> > > >
> > >> > > > Denis, which changes exactly? In current implementation of
> ticket
> > >> [2]
> > >> > > flag
> > >> > > > [1] is checked before requiring --force flag and showing any
> > >> warnings.
> > >> > Do
> > >> > > > we need to set reuse-memory-on-deactivate to true by default?
> > >> > > >
> > >> > > > [1]
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > >
> > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >> > > >
> > >> > > >
> > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > >> > > >
> > >> > > > > That's the best solution for this scenario. Should we readjust
> > the
> > >> > > > already
> > >> > > > > created ticket [1] suggesting to implement the changes of Alex
> > >> > > Scherbakov
> > >> > > > > instead?
> > >> > > > >
> > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > >> > > > >
> > >> > > > > -
> > >> > > > > Denis
> > >> > > > >
> > >> > > > >
> > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > >> > > > > alexey.scherbakoff@gmail.com> wrote:
> > >> > > > >
> > >> > > > > > Folks,
> > >> > > > > >
> > >> > > > > > For a long time we have a flag [1]
> > >> > > > > >
> > >> > > > > > It does almost what we want here.
> > >> > > > > >
> > >> > > > > > I suggest to make this behavior default and rework it to
> keep
> > >> data
> > >> > in
> > >> > > > > > memory as well (we already have special "recovery" mode for
> > >> > caches).
> > >> > > > > >
> > >> > > > > > [1]
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > >> > > > > alexey.goncharuk@gmail.com
> > >> > > > > > >:
> > >> > > > > >
> > >> > > > > > > I do not mind making this change if we explicitly and
> > clearly
> > >> > > define
> > >> > > > > what
> > >> > > > > > > 'new inactive state' means. What should happen if a
> > partition
> > >> is
> > >> > > lost
> > >> > > > > in
> > >> > > > > > > inactive state? What if such node joins back the cluster
> > >> after?
> > >> > > Etc.
> > >> > > > > > >
> > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <
> dmagda@apache.org
> > >:
> > >> > > > > > >
> > >> > > > > > > > Back up Ivan's opinion here. Initially, the
> > >> > > activation/deactivation
> > >> > > > > was
> > >> > > > > > > > created for the baseline topology designed for cases
> with
> > >> > native
> > >> > > > > > > > persistence. My thinking was that the mechanism itended
> to
> > >> > > prevent
> > >> > > > > data
> > >> > > > > > > > inconsistencies while nodes with data on the disk will
> be
> > in
> > >> > the
> > >> > > > > > process
> > >> > > > > > > of
> > >> > > > > > > > joining the cluster.
> > >> > > > > > > >
> > >> > > > > > > > Artem, could you please update the docs bringing this to
> > the
> > >> > > > > attention
> > >> > > > > > of
> > >> > > > > > > > the user community?
> > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > >> > > > > > > >
> > >> > > > > > > > AG, what if we don't purge data from memory at least for
> > the
> > >> > > caches
> > >> > > > > not
> > >> > > > > > > > backed by the native persistence? Is this a big deal? We
> > can
> > >> > > > > certainly
> > >> > > > > > > put
> > >> > > > > > > > this off by my guts feel we'll return to this question
> > >> sooner
> > >> > or
> > >> > > > > later.
> > >> > > > > > > >
> > >> > > > > > > > -
> > >> > > > > > > > Denis
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > >> > > > vololo100@gmail.com>
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > For me it looks like some coincidence effect. I
> > understand
> > >> > that
> > >> > > > we
> > >> > > > > > get
> > >> > > > > > > > > such behavior because deactivation works the same way
> as
> > >> for
> > >> > > > > > > > > persistent caches. Was cluster activation/deactivation
> > >> > designed
> > >> > > > and
> > >> > > > > > > > > described for in-memory caches? Current behavior
> sounds
> > >> for
> > >> > me
> > >> > > a
> > >> > > > as
> > >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> > >> purged
> > >> > all
> > >> > > > > their
> > >> > > > > > > > > data.
> > >> > > > > > > > >
> > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > >> > > > > > > > alexey.goncharuk@gmail.com
> > >> > > > > > > > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > > Because originally the sole purpose of deactivation
> > was
> > >> > > > resource
> > >> > > > > > > > > > deallocation.
> > >> > > > > > > > > >
> > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > >> > dmagda@apache.org
> > >> > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Such a revelation for me that data is purged from
> > RAM
> > >> if
> > >> > > > > someone
> > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember why
> > we
> > >> > > decided
> > >> > > > > to
> > >> > > > > > > > > implement
> > >> > > > > > > > > > > it this way initially?
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > -
> > >> > > > > > > > > > > Denis
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > >> > > > > > > > > > > wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > > I agree on CLI and JMX because those interfaces
> > can
> > >> be
> > >> > > used
> > >> > > > > by
> > >> > > > > > a
> > >> > > > > > > > > system
> > >> > > > > > > > > > > > administrator and can be invoked by mistake.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > As for the Java API, personally, I find it
> strange
> > >> to
> > >> > add
> > >> > > > > > 'force'
> > >> > > > > > > > or
> > >> > > > > > > > > > > > 'confirm' flags to it because it is very
> unlikely
> > >> that
> > >> > > such
> > >> > > > > an
> > >> > > > > > > > > invocation
> > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> > during
> > >> the
> > >> > > > > testing
> > >> > > > > > > > > phase and
> > >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> > flag
> > >> > > > anyways.
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > --
> > >> > > > > > > > > Best regards,
> > >> > > > > > > > > Ivan Pavlukhin
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > >
> > >> > > > > > Best regards,
> > >> > > > > > Alexei Scherbakov
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexei Scherbakov <al...@gmail.com>]

Vladimir,

IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from
clearing.
It allows allocated memory reuse on re-activation to avoid OS specific
issues if allocated heap is rather large.

You are right to create separate ticket for implementing required behavior.

чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vl...@gmail.com>:

> Or, is flag [1] is actual only for persistence mode? Related ticket [2] is
> completely about disabled persistence.
> If previous assumption is right, I think, we can involve flag [1] in ticket
> [2].
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> [2] https://issues.apache.org/jira/browse/IGNITE-12614
>
> чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:
>
> > Denis, Alexei,
> >
> > Regarding usage of flag
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > [1]
> >
> > When enabled, I think the following test should work. But it fails.
> >
> > //----------------------------------------------------------------
> >     @Test
> >     public void testDataPresent() throws Exception {
> >         System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
> >
> >         IgniteEx i = startGrid(0);
> >
> >         assertFalse(
> >
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
> >             .isPersistenceEnabled() );
> >
> >         String name = "non-persistent-cache";
> >
> >         i.createCache(name).put(1L, 1L);
> >
> >         assertEquals(1L, i.cache(name).get(1L));
> >
> >         i.cluster().state(ClusterState.INACTIVE);
> >         i.cluster().state(ClusterState.ACTIVE);
> >
> >         assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
> >     }
> > //----------------------------------------------------------------
> >
> >
> > Several notes:
> >
> > - IgniteCacheDatabaseSharedManager#reuseMemory
> >                   is true
> > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
> >      is called with shutdown == false
> > - PageMemoryNoStoreImpl#stop(booleam deallocate)
> >                      is called with deallocate == false
> >
> > But the cache from the test still has zero size after reactivation.
> >
> > Is flag [1] disabled by default because it is not implemented / doesn't
> > work? Do we need to skip it in current ticket and rise new one?
> >
> > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
> >
> >> I believe there might be a consistency-related reason why this flag was
> >> disabled by default for caches that store data in Ignite native
> >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
> >> this.
> >>
> >> As for the memory-only caches or caches backed up by a CacheStore such
> as
> >> an RDBMS, enabling of the flag should be harmless. Once we do this,
> we'll
> >> eliminate the need to load the data back into the cluster which can be a
> >> time-consuming operation depending on the data volume.
> >>
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> >> wrote:
> >>
> >> > Denis, but why reuse-data-on-deactivate was disabled by default? There
> >> > should be a reason for that. Any data consistency issues when node
> gets
> >> > activated anew? We may use both solutions because the flag can be
> >> switched
> >> > off again.
> >> >
> >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> >> >
> >> > > Hi Vladimir,
> >> > >
> >> > > Yes, I'm suggesting us to enable this flag by
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > default instead of introducing --force flag and showing any
> warnings.
> >> > >
> >> > > -
> >> > > Denis
> >> > >
> >> > >
> >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vladsz83@gmail.com
> >
> >> > > wrote:
> >> > >
> >> > > > Hello all.
> >> > > >
> >> > > > Denis, which changes exactly? In current implementation of ticket
> >> [2]
> >> > > flag
> >> > > > [1] is checked before requiring --force flag and showing any
> >> warnings.
> >> > Do
> >> > > > we need to set reuse-memory-on-deactivate to true by default?
> >> > > >
> >> > > > [1]
> >> > > >
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > >
> >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >> > > >
> >> > > >
> >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> >> > > >
> >> > > > > That's the best solution for this scenario. Should we readjust
> the
> >> > > > already
> >> > > > > created ticket [1] suggesting to implement the changes of Alex
> >> > > Scherbakov
> >> > > > > instead?
> >> > > > >
> >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >> > > > >
> >> > > > > -
> >> > > > > Denis
> >> > > > >
> >> > > > >
> >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> >> > > > > alexey.scherbakoff@gmail.com> wrote:
> >> > > > >
> >> > > > > > Folks,
> >> > > > > >
> >> > > > > > For a long time we have a flag [1]
> >> > > > > >
> >> > > > > > It does almost what we want here.
> >> > > > > >
> >> > > > > > I suggest to make this behavior default and rework it to keep
> >> data
> >> > in
> >> > > > > > memory as well (we already have special "recovery" mode for
> >> > caches).
> >> > > > > >
> >> > > > > > [1]
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> >> > > > > alexey.goncharuk@gmail.com
> >> > > > > > >:
> >> > > > > >
> >> > > > > > > I do not mind making this change if we explicitly and
> clearly
> >> > > define
> >> > > > > what
> >> > > > > > > 'new inactive state' means. What should happen if a
> partition
> >> is
> >> > > lost
> >> > > > > in
> >> > > > > > > inactive state? What if such node joins back the cluster
> >> after?
> >> > > Etc.
> >> > > > > > >
> >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dmagda@apache.org
> >:
> >> > > > > > >
> >> > > > > > > > Back up Ivan's opinion here. Initially, the
> >> > > activation/deactivation
> >> > > > > was
> >> > > > > > > > created for the baseline topology designed for cases with
> >> > native
> >> > > > > > > > persistence. My thinking was that the mechanism itended to
> >> > > prevent
> >> > > > > data
> >> > > > > > > > inconsistencies while nodes with data on the disk will be
> in
> >> > the
> >> > > > > > process
> >> > > > > > > of
> >> > > > > > > > joining the cluster.
> >> > > > > > > >
> >> > > > > > > > Artem, could you please update the docs bringing this to
> the
> >> > > > > attention
> >> > > > > > of
> >> > > > > > > > the user community?
> >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> >> > > > > > > >
> >> > > > > > > > AG, what if we don't purge data from memory at least for
> the
> >> > > caches
> >> > > > > not
> >> > > > > > > > backed by the native persistence? Is this a big deal? We
> can
> >> > > > > certainly
> >> > > > > > > put
> >> > > > > > > > this off by my guts feel we'll return to this question
> >> sooner
> >> > or
> >> > > > > later.
> >> > > > > > > >
> >> > > > > > > > -
> >> > > > > > > > Denis
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> >> > > > vololo100@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > For me it looks like some coincidence effect. I
> understand
> >> > that
> >> > > > we
> >> > > > > > get
> >> > > > > > > > > such behavior because deactivation works the same way as
> >> for
> >> > > > > > > > > persistent caches. Was cluster activation/deactivation
> >> > designed
> >> > > > and
> >> > > > > > > > > described for in-memory caches? Current behavior sounds
> >> for
> >> > me
> >> > > a
> >> > > > as
> >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
> >> purged
> >> > all
> >> > > > > their
> >> > > > > > > > > data.
> >> > > > > > > > >
> >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> >> > > > > > > > alexey.goncharuk@gmail.com
> >> > > > > > > > > >:
> >> > > > > > > > > >
> >> > > > > > > > > > Because originally the sole purpose of deactivation
> was
> >> > > > resource
> >> > > > > > > > > > deallocation.
> >> > > > > > > > > >
> >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> >> > dmagda@apache.org
> >> > > >:
> >> > > > > > > > > >
> >> > > > > > > > > > > Such a revelation for me that data is purged from
> RAM
> >> if
> >> > > > > someone
> >> > > > > > > > > > > deactivates the cluster. Alex, do you remember why
> we
> >> > > decided
> >> > > > > to
> >> > > > > > > > > implement
> >> > > > > > > > > > > it this way initially?
> >> > > > > > > > > > >
> >> > > > > > > > > > > -
> >> > > > > > > > > > > Denis
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> >> > > > > > > > > > > alexey.goncharuk@gmail.com>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > I agree on CLI and JMX because those interfaces
> can
> >> be
> >> > > used
> >> > > > > by
> >> > > > > > a
> >> > > > > > > > > system
> >> > > > > > > > > > > > administrator and can be invoked by mistake.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > As for the Java API, personally, I find it strange
> >> to
> >> > add
> >> > > > > > 'force'
> >> > > > > > > > or
> >> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
> >> that
> >> > > such
> >> > > > > an
> >> > > > > > > > > invocation
> >> > > > > > > > > > > > is done by mistake. Such mistakes are caught
> during
> >> the
> >> > > > > testing
> >> > > > > > > > > phase and
> >> > > > > > > > > > > > developers will end up hard-coding 'true' as a
> flag
> >> > > > anyways.
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > Best regards,
> >> > > > > > > > > Ivan Pavlukhin
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > >
> >> > > > > > Best regards,
> >> > > > > > Alexei Scherbakov
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>


-- 

Best regards,
Alexei Scherbakov

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Or, is flag [1] is actual only for persistence mode? Related ticket [2] is
completely about disabled persistence.
If previous assumption is right, I think, we can involve flag [1] in ticket
[2].

[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[2] https://issues.apache.org/jira/browse/IGNITE-12614

чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vl...@gmail.com>:

> Denis, Alexei,
>
> Regarding usage of flag
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> [1]
>
> When enabled, I think the following test should work. But it fails.
>
> //----------------------------------------------------------------
>     @Test
>     public void testDataPresent() throws Exception {
>         System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");
>
>         IgniteEx i = startGrid(0);
>
>         assertFalse(
> i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
>             .isPersistenceEnabled() );
>
>         String name = "non-persistent-cache";
>
>         i.createCache(name).put(1L, 1L);
>
>         assertEquals(1L, i.cache(name).get(1L));
>
>         i.cluster().state(ClusterState.INACTIVE);
>         i.cluster().state(ClusterState.ACTIVE);
>
>         assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
>     }
> //----------------------------------------------------------------
>
>
> Several notes:
>
> - IgniteCacheDatabaseSharedManager#reuseMemory
>                   is true
> - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
>      is called with shutdown == false
> - PageMemoryNoStoreImpl#stop(booleam deallocate)
>                      is called with deallocate == false
>
> But the cache from the test still has zero size after reactivation.
>
> Is flag [1] disabled by default because it is not implemented / doesn't
> work? Do we need to skip it in current ticket and rise new one?
>
> ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:
>
>> I believe there might be a consistency-related reason why this flag was
>> disabled by default for caches that store data in Ignite native
>> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
>> this.
>>
>> As for the memory-only caches or caches backed up by a CacheStore such as
>> an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
>> eliminate the need to load the data back into the cluster which can be a
>> time-consuming operation depending on the data volume.
>>
>>
>> -
>> Denis
>>
>>
>> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
>> wrote:
>>
>> > Denis, but why reuse-data-on-deactivate was disabled by default? There
>> > should be a reason for that. Any data consistency issues when node gets
>> > activated anew? We may use both solutions because the flag can be
>> switched
>> > off again.
>> >
>> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
>> >
>> > > Hi Vladimir,
>> > >
>> > > Yes, I'm suggesting us to enable this flag by
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > default instead of introducing --force flag and showing any warnings.
>> > >
>> > > -
>> > > Denis
>> > >
>> > >
>> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hello all.
>> > > >
>> > > > Denis, which changes exactly? In current implementation of ticket
>> [2]
>> > > flag
>> > > > [1] is checked before requiring --force flag and showing any
>> warnings.
>> > Do
>> > > > we need to set reuse-memory-on-deactivate to true by default?
>> > > >
>> > > > [1]
>> > > >
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > >
>> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
>> > > >
>> > > >
>> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
>> > > >
>> > > > > That's the best solution for this scenario. Should we readjust the
>> > > > already
>> > > > > created ticket [1] suggesting to implement the changes of Alex
>> > > Scherbakov
>> > > > > instead?
>> > > > >
>> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
>> > > > > alexey.scherbakoff@gmail.com> wrote:
>> > > > >
>> > > > > > Folks,
>> > > > > >
>> > > > > > For a long time we have a flag [1]
>> > > > > >
>> > > > > > It does almost what we want here.
>> > > > > >
>> > > > > > I suggest to make this behavior default and rework it to keep
>> data
>> > in
>> > > > > > memory as well (we already have special "recovery" mode for
>> > caches).
>> > > > > >
>> > > > > > [1]
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
>> > > > > alexey.goncharuk@gmail.com
>> > > > > > >:
>> > > > > >
>> > > > > > > I do not mind making this change if we explicitly and clearly
>> > > define
>> > > > > what
>> > > > > > > 'new inactive state' means. What should happen if a partition
>> is
>> > > lost
>> > > > > in
>> > > > > > > inactive state? What if such node joins back the cluster
>> after?
>> > > Etc.
>> > > > > > >
>> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>> > > > > > >
>> > > > > > > > Back up Ivan's opinion here. Initially, the
>> > > activation/deactivation
>> > > > > was
>> > > > > > > > created for the baseline topology designed for cases with
>> > native
>> > > > > > > > persistence. My thinking was that the mechanism itended to
>> > > prevent
>> > > > > data
>> > > > > > > > inconsistencies while nodes with data on the disk will be in
>> > the
>> > > > > > process
>> > > > > > > of
>> > > > > > > > joining the cluster.
>> > > > > > > >
>> > > > > > > > Artem, could you please update the docs bringing this to the
>> > > > > attention
>> > > > > > of
>> > > > > > > > the user community?
>> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
>> > > > > > > >
>> > > > > > > > AG, what if we don't purge data from memory at least for the
>> > > caches
>> > > > > not
>> > > > > > > > backed by the native persistence? Is this a big deal? We can
>> > > > > certainly
>> > > > > > > put
>> > > > > > > > this off by my guts feel we'll return to this question
>> sooner
>> > or
>> > > > > later.
>> > > > > > > >
>> > > > > > > > -
>> > > > > > > > Denis
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
>> > > > vololo100@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > For me it looks like some coincidence effect. I understand
>> > that
>> > > > we
>> > > > > > get
>> > > > > > > > > such behavior because deactivation works the same way as
>> for
>> > > > > > > > > persistent caches. Was cluster activation/deactivation
>> > designed
>> > > > and
>> > > > > > > > > described for in-memory caches? Current behavior sounds
>> for
>> > me
>> > > a
>> > > > as
>> > > > > > > > > big risk. I expect a lot of upset users unexpectedly
>> purged
>> > all
>> > > > > their
>> > > > > > > > > data.
>> > > > > > > > >
>> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
>> > > > > > > > alexey.goncharuk@gmail.com
>> > > > > > > > > >:
>> > > > > > > > > >
>> > > > > > > > > > Because originally the sole purpose of deactivation was
>> > > > resource
>> > > > > > > > > > deallocation.
>> > > > > > > > > >
>> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
>> > dmagda@apache.org
>> > > >:
>> > > > > > > > > >
>> > > > > > > > > > > Such a revelation for me that data is purged from RAM
>> if
>> > > > > someone
>> > > > > > > > > > > deactivates the cluster. Alex, do you remember why we
>> > > decided
>> > > > > to
>> > > > > > > > > implement
>> > > > > > > > > > > it this way initially?
>> > > > > > > > > > >
>> > > > > > > > > > > -
>> > > > > > > > > > > Denis
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
>> > > > > > > > > > > alexey.goncharuk@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > I agree on CLI and JMX because those interfaces can
>> be
>> > > used
>> > > > > by
>> > > > > > a
>> > > > > > > > > system
>> > > > > > > > > > > > administrator and can be invoked by mistake.
>> > > > > > > > > > > >
>> > > > > > > > > > > > As for the Java API, personally, I find it strange
>> to
>> > add
>> > > > > > 'force'
>> > > > > > > > or
>> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
>> that
>> > > such
>> > > > > an
>> > > > > > > > > invocation
>> > > > > > > > > > > > is done by mistake. Such mistakes are caught during
>> the
>> > > > > testing
>> > > > > > > > > phase and
>> > > > > > > > > > > > developers will end up hard-coding 'true' as a flag
>> > > > anyways.
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Best regards,
>> > > > > > > > > Ivan Pavlukhin
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > >
>> > > > > > Best regards,
>> > > > > > Alexei Scherbakov
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Denis, Alexei,

Regarding usage of flag
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
[1]

When enabled, I think the following test should work. But it fails.

//----------------------------------------------------------------
    @Test
    public void testDataPresent() throws Exception {
        System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, "true");

        IgniteEx i = startGrid(0);

        assertFalse(
i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration()
            .isPersistenceEnabled() );

        String name = "non-persistent-cache";

        i.createCache(name).put(1L, 1L);

        assertEquals(1L, i.cache(name).get(1L));

        i.cluster().state(ClusterState.INACTIVE);
        i.cluster().state(ClusterState.ACTIVE);

        assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
    }
//----------------------------------------------------------------


Several notes:

- IgniteCacheDatabaseSharedManager#reuseMemory
                is true
- IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown)
     is called with shutdown == false
- PageMemoryNoStoreImpl#stop(booleam deallocate)
                   is called with deallocate == false

But the cache from the test still has zero size after reactivation.

Is flag [1] disabled by default because it is not implemented / doesn't
work? Do we need to skip it in current ticket and rise new one?

ср, 5 февр. 2020 г. в 21:05, Denis Magda <dm...@apache.org>:

> I believe there might be a consistency-related reason why this flag was
> disabled by default for caches that store data in Ignite native
> persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
> this.
>
> As for the memory-only caches or caches backed up by a CacheStore such as
> an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
> eliminate the need to load the data back into the cluster which can be a
> time-consuming operation depending on the data volume.
>
>
> -
> Denis
>
>
> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com>
> wrote:
>
> > Denis, but why reuse-data-on-deactivate was disabled by default? There
> > should be a reason for that. Any data consistency issues when node gets
> > activated anew? We may use both solutions because the flag can be
> switched
> > off again.
> >
> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
> >
> > > Hi Vladimir,
> > >
> > > Yes, I'm suggesting us to enable this flag by
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > default instead of introducing --force flag and showing any warnings.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> > > wrote:
> > >
> > > > Hello all.
> > > >
> > > > Denis, which changes exactly? In current implementation of ticket [2]
> > > flag
> > > > [1] is checked before requiring --force flag and showing any
> warnings.
> > Do
> > > > we need to set reuse-memory-on-deactivate to true by default?
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > >
> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > > >
> > > > > That's the best solution for this scenario. Should we readjust the
> > > > already
> > > > > created ticket [1] suggesting to implement the changes of Alex
> > > Scherbakov
> > > > > instead?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > > alexey.scherbakoff@gmail.com> wrote:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > For a long time we have a flag [1]
> > > > > >
> > > > > > It does almost what we want here.
> > > > > >
> > > > > > I suggest to make this behavior default and rework it to keep
> data
> > in
> > > > > > memory as well (we already have special "recovery" mode for
> > caches).
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > > >
> > > > > >
> > > > > >
> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > > >:
> > > > > >
> > > > > > > I do not mind making this change if we explicitly and clearly
> > > define
> > > > > what
> > > > > > > 'new inactive state' means. What should happen if a partition
> is
> > > lost
> > > > > in
> > > > > > > inactive state? What if such node joins back the cluster after?
> > > Etc.
> > > > > > >
> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > > > >
> > > > > > > > Back up Ivan's opinion here. Initially, the
> > > activation/deactivation
> > > > > was
> > > > > > > > created for the baseline topology designed for cases with
> > native
> > > > > > > > persistence. My thinking was that the mechanism itended to
> > > prevent
> > > > > data
> > > > > > > > inconsistencies while nodes with data on the disk will be in
> > the
> > > > > > process
> > > > > > > of
> > > > > > > > joining the cluster.
> > > > > > > >
> > > > > > > > Artem, could you please update the docs bringing this to the
> > > > > attention
> > > > > > of
> > > > > > > > the user community?
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > > > >
> > > > > > > > AG, what if we don't purge data from memory at least for the
> > > caches
> > > > > not
> > > > > > > > backed by the native persistence? Is this a big deal? We can
> > > > > certainly
> > > > > > > put
> > > > > > > > this off by my guts feel we'll return to this question sooner
> > or
> > > > > later.
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > > vololo100@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > For me it looks like some coincidence effect. I understand
> > that
> > > > we
> > > > > > get
> > > > > > > > > such behavior because deactivation works the same way as
> for
> > > > > > > > > persistent caches. Was cluster activation/deactivation
> > designed
> > > > and
> > > > > > > > > described for in-memory caches? Current behavior sounds for
> > me
> > > a
> > > > as
> > > > > > > > > big risk. I expect a lot of upset users unexpectedly purged
> > all
> > > > > their
> > > > > > > > > data.
> > > > > > > > >
> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > Because originally the sole purpose of deactivation was
> > > > resource
> > > > > > > > > > deallocation.
> > > > > > > > > >
> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> > dmagda@apache.org
> > > >:
> > > > > > > > > >
> > > > > > > > > > > Such a revelation for me that data is purged from RAM
> if
> > > > > someone
> > > > > > > > > > > deactivates the cluster. Alex, do you remember why we
> > > decided
> > > > > to
> > > > > > > > > implement
> > > > > > > > > > > it this way initially?
> > > > > > > > > > >
> > > > > > > > > > > -
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I agree on CLI and JMX because those interfaces can
> be
> > > used
> > > > > by
> > > > > > a
> > > > > > > > > system
> > > > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > > > >
> > > > > > > > > > > > As for the Java API, personally, I find it strange to
> > add
> > > > > > 'force'
> > > > > > > > or
> > > > > > > > > > > > 'confirm' flags to it because it is very unlikely
> that
> > > such
> > > > > an
> > > > > > > > > invocation
> > > > > > > > > > > > is done by mistake. Such mistakes are caught during
> the
> > > > > testing
> > > > > > > > > phase and
> > > > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > > > anyways.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Ivan Pavlukhin
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best regards,
> > > > > > Alexei Scherbakov
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Denis Magda <dm...@apache.org>]

I believe there might be a consistency-related reason why this flag was
disabled by default for caches that store data in Ignite native
persistence. I hope, Alex Goncharuk or Scherbakov can shed some light on
this.

As for the memory-only caches or caches backed up by a CacheStore such as
an RDBMS, enabling of the flag should be harmless. Once we do this, we'll
eliminate the need to load the data back into the cluster which can be a
time-consuming operation depending on the data volume.


-
Denis


On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin <vl...@gmail.com> wrote:

> Denis, but why reuse-data-on-deactivate was disabled by default? There
> should be a reason for that. Any data consistency issues when node gets
> activated anew? We may use both solutions because the flag can be switched
> off again.
>
> ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:
>
> > Hi Vladimir,
> >
> > Yes, I'm suggesting us to enable this flag by
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > default instead of introducing --force flag and showing any warnings.
> >
> > -
> > Denis
> >
> >
> > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> > wrote:
> >
> > > Hello all.
> > >
> > > Denis, which changes exactly? In current implementation of ticket [2]
> > flag
> > > [1] is checked before requiring --force flag and showing any warnings.
> Do
> > > we need to set reuse-memory-on-deactivate to true by default?
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >
> > > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > >
> > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> > >
> > > > That's the best solution for this scenario. Should we readjust the
> > > already
> > > > created ticket [1] suggesting to implement the changes of Alex
> > Scherbakov
> > > > instead?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > > alexey.scherbakoff@gmail.com> wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > For a long time we have a flag [1]
> > > > >
> > > > > It does almost what we want here.
> > > > >
> > > > > I suggest to make this behavior default and rework it to keep data
> in
> > > > > memory as well (we already have special "recovery" mode for
> caches).
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > > >
> > > > >
> > > > >
> > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > > >:
> > > > >
> > > > > > I do not mind making this change if we explicitly and clearly
> > define
> > > > what
> > > > > > 'new inactive state' means. What should happen if a partition is
> > lost
> > > > in
> > > > > > inactive state? What if such node joins back the cluster after?
> > Etc.
> > > > > >
> > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > > > Back up Ivan's opinion here. Initially, the
> > activation/deactivation
> > > > was
> > > > > > > created for the baseline topology designed for cases with
> native
> > > > > > > persistence. My thinking was that the mechanism itended to
> > prevent
> > > > data
> > > > > > > inconsistencies while nodes with data on the disk will be in
> the
> > > > > process
> > > > > > of
> > > > > > > joining the cluster.
> > > > > > >
> > > > > > > Artem, could you please update the docs bringing this to the
> > > > attention
> > > > > of
> > > > > > > the user community?
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > > >
> > > > > > > AG, what if we don't purge data from memory at least for the
> > caches
> > > > not
> > > > > > > backed by the native persistence? Is this a big deal? We can
> > > > certainly
> > > > > > put
> > > > > > > this off by my guts feel we'll return to this question sooner
> or
> > > > later.
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > > vololo100@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > For me it looks like some coincidence effect. I understand
> that
> > > we
> > > > > get
> > > > > > > > such behavior because deactivation works the same way as for
> > > > > > > > persistent caches. Was cluster activation/deactivation
> designed
> > > and
> > > > > > > > described for in-memory caches? Current behavior sounds for
> me
> > a
> > > as
> > > > > > > > big risk. I expect a lot of upset users unexpectedly purged
> all
> > > > their
> > > > > > > > data.
> > > > > > > >
> > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com
> > > > > > > > >:
> > > > > > > > >
> > > > > > > > > Because originally the sole purpose of deactivation was
> > > resource
> > > > > > > > > deallocation.
> > > > > > > > >
> > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <
> dmagda@apache.org
> > >:
> > > > > > > > >
> > > > > > > > > > Such a revelation for me that data is purged from RAM if
> > > > someone
> > > > > > > > > > deactivates the cluster. Alex, do you remember why we
> > decided
> > > > to
> > > > > > > > implement
> > > > > > > > > > it this way initially?
> > > > > > > > > >
> > > > > > > > > > -
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I agree on CLI and JMX because those interfaces can be
> > used
> > > > by
> > > > > a
> > > > > > > > system
> > > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > > >
> > > > > > > > > > > As for the Java API, personally, I find it strange to
> add
> > > > > 'force'
> > > > > > > or
> > > > > > > > > > > 'confirm' flags to it because it is very unlikely that
> > such
> > > > an
> > > > > > > > invocation
> > > > > > > > > > > is done by mistake. Such mistakes are caught during the
> > > > testing
> > > > > > > > phase and
> > > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > > anyways.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Ivan Pavlukhin
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > Alexei Scherbakov
> > > > >
> > > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Denis, but why reuse-data-on-deactivate was disabled by default? There
should be a reason for that. Any data consistency issues when node gets
activated anew? We may use both solutions because the flag can be switched
off again.

ср, 5 февр. 2020 г. в 20:47, Denis Magda <dm...@apache.org>:

> Hi Vladimir,
>
> Yes, I'm suggesting us to enable this flag by
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> default instead of introducing --force flag and showing any warnings.
>
> -
> Denis
>
>
> On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com>
> wrote:
>
> > Hello all.
> >
> > Denis, which changes exactly? In current implementation of ticket [2]
> flag
> > [1] is checked before requiring --force flag and showing any warnings. Do
> > we need to set reuse-memory-on-deactivate to true by default?
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >
> > [2] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> >
> > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
> >
> > > That's the best solution for this scenario. Should we readjust the
> > already
> > > created ticket [1] suggesting to implement the changes of Alex
> Scherbakov
> > > instead?
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > > alexey.scherbakoff@gmail.com> wrote:
> > >
> > > > Folks,
> > > >
> > > > For a long time we have a flag [1]
> > > >
> > > > It does almost what we want here.
> > > >
> > > > I suggest to make this behavior default and rework it to keep data in
> > > > memory as well (we already have special "recovery" mode for caches).
> > > >
> > > > [1]
> > > >
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > > >
> > > >
> > > >
> > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >:
> > > >
> > > > > I do not mind making this change if we explicitly and clearly
> define
> > > what
> > > > > 'new inactive state' means. What should happen if a partition is
> lost
> > > in
> > > > > inactive state? What if such node joins back the cluster after?
> Etc.
> > > > >
> > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > > >
> > > > > > Back up Ivan's opinion here. Initially, the
> activation/deactivation
> > > was
> > > > > > created for the baseline topology designed for cases with native
> > > > > > persistence. My thinking was that the mechanism itended to
> prevent
> > > data
> > > > > > inconsistencies while nodes with data on the disk will be in the
> > > > process
> > > > > of
> > > > > > joining the cluster.
> > > > > >
> > > > > > Artem, could you please update the docs bringing this to the
> > > attention
> > > > of
> > > > > > the user community?
> > > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > > >
> > > > > > AG, what if we don't purge data from memory at least for the
> caches
> > > not
> > > > > > backed by the native persistence? Is this a big deal? We can
> > > certainly
> > > > > put
> > > > > > this off by my guts feel we'll return to this question sooner or
> > > later.
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> > vololo100@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > For me it looks like some coincidence effect. I understand that
> > we
> > > > get
> > > > > > > such behavior because deactivation works the same way as for
> > > > > > > persistent caches. Was cluster activation/deactivation designed
> > and
> > > > > > > described for in-memory caches? Current behavior sounds for me
> a
> > as
> > > > > > > big risk. I expect a lot of upset users unexpectedly purged all
> > > their
> > > > > > > data.
> > > > > > >
> > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com
> > > > > > > >:
> > > > > > > >
> > > > > > > > Because originally the sole purpose of deactivation was
> > resource
> > > > > > > > deallocation.
> > > > > > > >
> > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dmagda@apache.org
> >:
> > > > > > > >
> > > > > > > > > Such a revelation for me that data is purged from RAM if
> > > someone
> > > > > > > > > deactivates the cluster. Alex, do you remember why we
> decided
> > > to
> > > > > > > implement
> > > > > > > > > it this way initially?
> > > > > > > > >
> > > > > > > > > -
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I agree on CLI and JMX because those interfaces can be
> used
> > > by
> > > > a
> > > > > > > system
> > > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > > >
> > > > > > > > > > As for the Java API, personally, I find it strange to add
> > > > 'force'
> > > > > > or
> > > > > > > > > > 'confirm' flags to it because it is very unlikely that
> such
> > > an
> > > > > > > invocation
> > > > > > > > > > is done by mistake. Such mistakes are caught during the
> > > testing
> > > > > > > phase and
> > > > > > > > > > developers will end up hard-coding 'true' as a flag
> > anyways.
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Alexei Scherbakov
> > > >
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Denis Magda <dm...@apache.org>]

Hi Vladimir,

Yes, I'm suggesting us to enable this flag by
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
default instead of introducing --force flag and showing any warnings.

-
Denis


On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin <vl...@gmail.com> wrote:

> Hello all.
>
> Denis, which changes exactly? In current implementation of ticket [2] flag
> [1] is checked before requiring --force flag and showing any warnings. Do
> we need to set reuse-memory-on-deactivate to true by default?
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
> [2] https://issues.apache.org/jira/browse/IGNITE-12614
>
>
> вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:
>
> > That's the best solution for this scenario. Should we readjust the
> already
> > created ticket [1] suggesting to implement the changes of Alex Scherbakov
> > instead?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12614
> >
> > -
> > Denis
> >
> >
> > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> > alexey.scherbakoff@gmail.com> wrote:
> >
> > > Folks,
> > >
> > > For a long time we have a flag [1]
> > >
> > > It does almost what we want here.
> > >
> > > I suggest to make this behavior default and rework it to keep data in
> > > memory as well (we already have special "recovery" mode for caches).
> > >
> > > [1]
> > >
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> > >
> > >
> > >
> > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > >
> > > > I do not mind making this change if we explicitly and clearly define
> > what
> > > > 'new inactive state' means. What should happen if a partition is lost
> > in
> > > > inactive state? What if such node joins back the cluster after? Etc.
> > > >
> > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > > >
> > > > > Back up Ivan's opinion here. Initially, the activation/deactivation
> > was
> > > > > created for the baseline topology designed for cases with native
> > > > > persistence. My thinking was that the mechanism itended to prevent
> > data
> > > > > inconsistencies while nodes with data on the disk will be in the
> > > process
> > > > of
> > > > > joining the cluster.
> > > > >
> > > > > Artem, could you please update the docs bringing this to the
> > attention
> > > of
> > > > > the user community?
> > > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > > >
> > > > > AG, what if we don't purge data from memory at least for the caches
> > not
> > > > > backed by the native persistence? Is this a big deal? We can
> > certainly
> > > > put
> > > > > this off by my guts feel we'll return to this question sooner or
> > later.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <
> vololo100@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > For me it looks like some coincidence effect. I understand that
> we
> > > get
> > > > > > such behavior because deactivation works the same way as for
> > > > > > persistent caches. Was cluster activation/deactivation designed
> and
> > > > > > described for in-memory caches? Current behavior sounds for me a
> as
> > > > > > big risk. I expect a lot of upset users unexpectedly purged all
> > their
> > > > > > data.
> > > > > >
> > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > > >:
> > > > > > >
> > > > > > > Because originally the sole purpose of deactivation was
> resource
> > > > > > > deallocation.
> > > > > > >
> > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > > > >
> > > > > > > > Such a revelation for me that data is purged from RAM if
> > someone
> > > > > > > > deactivates the cluster. Alex, do you remember why we decided
> > to
> > > > > > implement
> > > > > > > > it this way initially?
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I agree on CLI and JMX because those interfaces can be used
> > by
> > > a
> > > > > > system
> > > > > > > > > administrator and can be invoked by mistake.
> > > > > > > > >
> > > > > > > > > As for the Java API, personally, I find it strange to add
> > > 'force'
> > > > > or
> > > > > > > > > 'confirm' flags to it because it is very unlikely that such
> > an
> > > > > > invocation
> > > > > > > > > is done by mistake. Such mistakes are caught during the
> > testing
> > > > > > phase and
> > > > > > > > > developers will end up hard-coding 'true' as a flag
> anyways.
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Vladimir Steshin <vl...@gmail.com>]

Hello all.

Denis, which changes exactly? In current implementation of ticket [2] flag
[1] is checked before requiring --force flag and showing any warnings. Do
we need to set reuse-memory-on-deactivate to true by default?

[1]
org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE

[2] https://issues.apache.org/jira/browse/IGNITE-12614


вт, 4 февр. 2020 г. в 22:45, Denis Magda <dm...@apache.org>:

> That's the best solution for this scenario. Should we readjust the already
> created ticket [1] suggesting to implement the changes of Alex Scherbakov
> instead?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12614
>
> -
> Denis
>
>
> On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> wrote:
>
> > Folks,
> >
> > For a long time we have a flag [1]
> >
> > It does almost what we want here.
> >
> > I suggest to make this behavior default and rework it to keep data in
> > memory as well (we already have special "recovery" mode for caches).
> >
> > [1]
> >
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
> >
> >
> >
> > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > I do not mind making this change if we explicitly and clearly define
> what
> > > 'new inactive state' means. What should happen if a partition is lost
> in
> > > inactive state? What if such node joins back the cluster after? Etc.
> > >
> > > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> > >
> > > > Back up Ivan's opinion here. Initially, the activation/deactivation
> was
> > > > created for the baseline topology designed for cases with native
> > > > persistence. My thinking was that the mechanism itended to prevent
> data
> > > > inconsistencies while nodes with data on the disk will be in the
> > process
> > > of
> > > > joining the cluster.
> > > >
> > > > Artem, could you please update the docs bringing this to the
> attention
> > of
> > > > the user community?
> > > > https://issues.apache.org/jira/browse/IGNITE-12615
> > > >
> > > > AG, what if we don't purge data from memory at least for the caches
> not
> > > > backed by the native persistence? Is this a big deal? We can
> certainly
> > > put
> > > > this off by my guts feel we'll return to this question sooner or
> later.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > > > wrote:
> > > >
> > > > > For me it looks like some coincidence effect. I understand that we
> > get
> > > > > such behavior because deactivation works the same way as for
> > > > > persistent caches. Was cluster activation/deactivation designed and
> > > > > described for in-memory caches? Current behavior sounds for me a as
> > > > > big risk. I expect a lot of upset users unexpectedly purged all
> their
> > > > > data.
> > > > >
> > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > > >:
> > > > > >
> > > > > > Because originally the sole purpose of deactivation was resource
> > > > > > deallocation.
> > > > > >
> > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > > > Such a revelation for me that data is purged from RAM if
> someone
> > > > > > > deactivates the cluster. Alex, do you remember why we decided
> to
> > > > > implement
> > > > > > > it this way initially?
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I agree on CLI and JMX because those interfaces can be used
> by
> > a
> > > > > system
> > > > > > > > administrator and can be invoked by mistake.
> > > > > > > >
> > > > > > > > As for the Java API, personally, I find it strange to add
> > 'force'
> > > > or
> > > > > > > > 'confirm' flags to it because it is very unlikely that such
> an
> > > > > invocation
> > > > > > > > is done by mistake. Such mistakes are caught during the
> testing
> > > > > phase and
> > > > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Denis Magda <dm...@apache.org>]

That's the best solution for this scenario. Should we readjust the already
created ticket [1] suggesting to implement the changes of Alex Scherbakov
instead?

[1] https://issues.apache.org/jira/browse/IGNITE-12614

-
Denis


On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov <
alexey.scherbakoff@gmail.com> wrote:

> Folks,
>
> For a long time we have a flag [1]
>
> It does almost what we want here.
>
> I suggest to make this behavior default and rework it to keep data in
> memory as well (we already have special "recovery" mode for caches).
>
> [1]
> org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
>
>
> пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > I do not mind making this change if we explicitly and clearly define what
> > 'new inactive state' means. What should happen if a partition is lost in
> > inactive state? What if such node joins back the cluster after? Etc.
> >
> > пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
> >
> > > Back up Ivan's opinion here. Initially, the activation/deactivation was
> > > created for the baseline topology designed for cases with native
> > > persistence. My thinking was that the mechanism itended to prevent data
> > > inconsistencies while nodes with data on the disk will be in the
> process
> > of
> > > joining the cluster.
> > >
> > > Artem, could you please update the docs bringing this to the attention
> of
> > > the user community?
> > > https://issues.apache.org/jira/browse/IGNITE-12615
> > >
> > > AG, what if we don't purge data from memory at least for the caches not
> > > backed by the native persistence? Is this a big deal? We can certainly
> > put
> > > this off by my guts feel we'll return to this question sooner or later.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > > wrote:
> > >
> > > > For me it looks like some coincidence effect. I understand that we
> get
> > > > such behavior because deactivation works the same way as for
> > > > persistent caches. Was cluster activation/deactivation designed and
> > > > described for in-memory caches? Current behavior sounds for me a as
> > > > big risk. I expect a lot of upset users unexpectedly purged all their
> > > > data.
> > > >
> > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >:
> > > > >
> > > > > Because originally the sole purpose of deactivation was resource
> > > > > deallocation.
> > > > >
> > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > > >
> > > > > > Such a revelation for me that data is purged from RAM if someone
> > > > > > deactivates the cluster. Alex, do you remember why we decided to
> > > > implement
> > > > > > it this way initially?
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I agree on CLI and JMX because those interfaces can be used by
> a
> > > > system
> > > > > > > administrator and can be invoked by mistake.
> > > > > > >
> > > > > > > As for the Java API, personally, I find it strange to add
> 'force'
> > > or
> > > > > > > 'confirm' flags to it because it is very unlikely that such an
> > > > invocation
> > > > > > > is done by mistake. Such mistakes are caught during the testing
> > > > phase and
> > > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexei Scherbakov <al...@gmail.com>]

Alexey Goncharuk,

I see no issues with "lost" partitions. The same is possible right now in
persistence mode. Node can be killed and data erased while grid is
deactivated.

After re-join it should rebalance itself from existing owners (if any
exists).

вт, 4 февр. 2020 г. в 10:54, Alexei Scherbakov <alexey.scherbakoff@gmail.com
>:

> Folks,
>
> For a long time we have a flag [1]
>
> It does almost what we want here.
>
> I suggest to make this behavior default and rework it to keep data in
> memory as well (we already have special "recovery" mode for caches).
>
>
> [1] org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE
>
>
>
> пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
>> I do not mind making this change if we explicitly and clearly define what
>> 'new inactive state' means. What should happen if a partition is lost in
>> inactive state? What if such node joins back the cluster after? Etc.
>>
>> пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>>
>> > Back up Ivan's opinion here. Initially, the activation/deactivation was
>> > created for the baseline topology designed for cases with native
>> > persistence. My thinking was that the mechanism itended to prevent data
>> > inconsistencies while nodes with data on the disk will be in the
>> process of
>> > joining the cluster.
>> >
>> > Artem, could you please update the docs bringing this to the attention
>> of
>> > the user community?
>> > https://issues.apache.org/jira/browse/IGNITE-12615
>> >
>> > AG, what if we don't purge data from memory at least for the caches not
>> > backed by the native persistence? Is this a big deal? We can certainly
>> put
>> > this off by my guts feel we'll return to this question sooner or later.
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
>> > wrote:
>> >
>> > > For me it looks like some coincidence effect. I understand that we get
>> > > such behavior because deactivation works the same way as for
>> > > persistent caches. Was cluster activation/deactivation designed and
>> > > described for in-memory caches? Current behavior sounds for me a as
>> > > big risk. I expect a lot of upset users unexpectedly purged all their
>> > > data.
>> > >
>> > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
>> > alexey.goncharuk@gmail.com
>> > > >:
>> > > >
>> > > > Because originally the sole purpose of deactivation was resource
>> > > > deallocation.
>> > > >
>> > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
>> > > >
>> > > > > Such a revelation for me that data is purged from RAM if someone
>> > > > > deactivates the cluster. Alex, do you remember why we decided to
>> > > implement
>> > > > > it this way initially?
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
>> > > > > alexey.goncharuk@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I agree on CLI and JMX because those interfaces can be used by a
>> > > system
>> > > > > > administrator and can be invoked by mistake.
>> > > > > >
>> > > > > > As for the Java API, personally, I find it strange to add
>> 'force'
>> > or
>> > > > > > 'confirm' flags to it because it is very unlikely that such an
>> > > invocation
>> > > > > > is done by mistake. Such mistakes are caught during the testing
>> > > phase and
>> > > > > > developers will end up hard-coding 'true' as a flag anyways.
>> > > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > > Ivan Pavlukhin
>> > >
>> >
>>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>


-- 

Best regards,
Alexei Scherbakov

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexei Scherbakov <al...@gmail.com>]

Folks,

For a long time we have a flag [1]

It does almost what we want here.

I suggest to make this behavior default and rework it to keep data in
memory as well (we already have special "recovery" mode for caches).

[1] org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE



пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk <al...@gmail.com>:

> I do not mind making this change if we explicitly and clearly define what
> 'new inactive state' means. What should happen if a partition is lost in
> inactive state? What if such node joins back the cluster after? Etc.
>
> пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:
>
> > Back up Ivan's opinion here. Initially, the activation/deactivation was
> > created for the baseline topology designed for cases with native
> > persistence. My thinking was that the mechanism itended to prevent data
> > inconsistencies while nodes with data on the disk will be in the process
> of
> > joining the cluster.
> >
> > Artem, could you please update the docs bringing this to the attention of
> > the user community?
> > https://issues.apache.org/jira/browse/IGNITE-12615
> >
> > AG, what if we don't purge data from memory at least for the caches not
> > backed by the native persistence? Is this a big deal? We can certainly
> put
> > this off by my guts feel we'll return to this question sooner or later.
> >
> > -
> > Denis
> >
> >
> > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> > wrote:
> >
> > > For me it looks like some coincidence effect. I understand that we get
> > > such behavior because deactivation works the same way as for
> > > persistent caches. Was cluster activation/deactivation designed and
> > > described for in-memory caches? Current behavior sounds for me a as
> > > big risk. I expect a lot of upset users unexpectedly purged all their
> > > data.
> > >
> > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > > >
> > > > Because originally the sole purpose of deactivation was resource
> > > > deallocation.
> > > >
> > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > > >
> > > > > Such a revelation for me that data is purged from RAM if someone
> > > > > deactivates the cluster. Alex, do you remember why we decided to
> > > implement
> > > > > it this way initially?
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I agree on CLI and JMX because those interfaces can be used by a
> > > system
> > > > > > administrator and can be invoked by mistake.
> > > > > >
> > > > > > As for the Java API, personally, I find it strange to add 'force'
> > or
> > > > > > 'confirm' flags to it because it is very unlikely that such an
> > > invocation
> > > > > > is done by mistake. Such mistakes are caught during the testing
> > > phase and
> > > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >
>


-- 

Best regards,
Alexei Scherbakov

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

I do not mind making this change if we explicitly and clearly define what
'new inactive state' means. What should happen if a partition is lost in
inactive state? What if such node joins back the cluster after? Etc.

пт, 31 янв. 2020 г. в 20:57, Denis Magda <dm...@apache.org>:

> Back up Ivan's opinion here. Initially, the activation/deactivation was
> created for the baseline topology designed for cases with native
> persistence. My thinking was that the mechanism itended to prevent data
> inconsistencies while nodes with data on the disk will be in the process of
> joining the cluster.
>
> Artem, could you please update the docs bringing this to the attention of
> the user community?
> https://issues.apache.org/jira/browse/IGNITE-12615
>
> AG, what if we don't purge data from memory at least for the caches not
> backed by the native persistence? Is this a big deal? We can certainly put
> this off by my guts feel we'll return to this question sooner or later.
>
> -
> Denis
>
>
> On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
> > For me it looks like some coincidence effect. I understand that we get
> > such behavior because deactivation works the same way as for
> > persistent caches. Was cluster activation/deactivation designed and
> > described for in-memory caches? Current behavior sounds for me a as
> > big risk. I expect a lot of upset users unexpectedly purged all their
> > data.
> >
> > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> > >
> > > Because originally the sole purpose of deactivation was resource
> > > deallocation.
> > >
> > > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> > >
> > > > Such a revelation for me that data is purged from RAM if someone
> > > > deactivates the cluster. Alex, do you remember why we decided to
> > implement
> > > > it this way initially?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com>
> > > > wrote:
> > > >
> > > > > I agree on CLI and JMX because those interfaces can be used by a
> > system
> > > > > administrator and can be invoked by mistake.
> > > > >
> > > > > As for the Java API, personally, I find it strange to add 'force'
> or
> > > > > 'confirm' flags to it because it is very unlikely that such an
> > invocation
> > > > > is done by mistake. Such mistakes are caught during the testing
> > phase and
> > > > > developers will end up hard-coding 'true' as a flag anyways.
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Denis Magda <dm...@apache.org>]

Back up Ivan's opinion here. Initially, the activation/deactivation was
created for the baseline topology designed for cases with native
persistence. My thinking was that the mechanism itended to prevent data
inconsistencies while nodes with data on the disk will be in the process of
joining the cluster.

Artem, could you please update the docs bringing this to the attention of
the user community?
https://issues.apache.org/jira/browse/IGNITE-12615

AG, what if we don't purge data from memory at least for the caches not
backed by the native persistence? Is this a big deal? We can certainly put
this off by my guts feel we'll return to this question sooner or later.

-
Denis


On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin <vo...@gmail.com> wrote:

> For me it looks like some coincidence effect. I understand that we get
> such behavior because deactivation works the same way as for
> persistent caches. Was cluster activation/deactivation designed and
> described for in-memory caches? Current behavior sounds for me a as
> big risk. I expect a lot of upset users unexpectedly purged all their
> data.
>
> пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > Because originally the sole purpose of deactivation was resource
> > deallocation.
> >
> > чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
> >
> > > Such a revelation for me that data is purged from RAM if someone
> > > deactivates the cluster. Alex, do you remember why we decided to
> implement
> > > it this way initially?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com>
> > > wrote:
> > >
> > > > I agree on CLI and JMX because those interfaces can be used by a
> system
> > > > administrator and can be invoked by mistake.
> > > >
> > > > As for the Java API, personally, I find it strange to add 'force' or
> > > > 'confirm' flags to it because it is very unlikely that such an
> invocation
> > > > is done by mistake. Such mistakes are caught during the testing
> phase and
> > > > developers will end up hard-coding 'true' as a flag anyways.
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Ivan Pavlukhin <vo...@gmail.com>]

For me it looks like some coincidence effect. I understand that we get
such behavior because deactivation works the same way as for
persistent caches. Was cluster activation/deactivation designed and
described for in-memory caches? Current behavior sounds for me a as
big risk. I expect a lot of upset users unexpectedly purged all their
data.

пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk <al...@gmail.com>:
>
> Because originally the sole purpose of deactivation was resource
> deallocation.
>
> чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:
>
> > Such a revelation for me that data is purged from RAM if someone
> > deactivates the cluster. Alex, do you remember why we decided to implement
> > it this way initially?
> >
> > -
> > Denis
> >
> >
> > On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> > alexey.goncharuk@gmail.com>
> > wrote:
> >
> > > I agree on CLI and JMX because those interfaces can be used by a system
> > > administrator and can be invoked by mistake.
> > >
> > > As for the Java API, personally, I find it strange to add 'force' or
> > > 'confirm' flags to it because it is very unlikely that such an invocation
> > > is done by mistake. Such mistakes are caught during the testing phase and
> > > developers will end up hard-coding 'true' as a flag anyways.
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

Because originally the sole purpose of deactivation was resource
deallocation.

чт, 30 янв. 2020 г. в 22:13, Denis Magda <dm...@apache.org>:

> Such a revelation for me that data is purged from RAM if someone
> deactivates the cluster. Alex, do you remember why we decided to implement
> it this way initially?
>
> -
> Denis
>
>
> On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> wrote:
>
> > I agree on CLI and JMX because those interfaces can be used by a system
> > administrator and can be invoked by mistake.
> >
> > As for the Java API, personally, I find it strange to add 'force' or
> > 'confirm' flags to it because it is very unlikely that such an invocation
> > is done by mistake. Such mistakes are caught during the testing phase and
> > developers will end up hard-coding 'true' as a flag anyways.
> >
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Denis Magda <dm...@apache.org>]

Such a revelation for me that data is purged from RAM if someone
deactivates the cluster. Alex, do you remember why we decided to implement
it this way initially?

-
Denis


On Thu, Jan 30, 2020 at 2:09 AM Alexey Goncharuk <al...@gmail.com>
wrote:

> I agree on CLI and JMX because those interfaces can be used by a system
> administrator and can be invoked by mistake.
>
> As for the Java API, personally, I find it strange to add 'force' or
> 'confirm' flags to it because it is very unlikely that such an invocation
> is done by mistake. Such mistakes are caught during the testing phase and
> developers will end up hard-coding 'true' as a flag anyways.
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

I agree on CLI and JMX because those interfaces can be used by a system
administrator and can be invoked by mistake.

As for the Java API, personally, I find it strange to add 'force' or
'confirm' flags to it because it is very unlikely that such an invocation
is done by mistake. Such mistakes are caught during the testing phase and
developers will end up hard-coding 'true' as a flag anyways.

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

As far as I know we have several places to manage cluster state:

1. Java API - Ignite#cluster#state
2. CLI utility - control.sh
3. JMX beans(can we manage cluster state from JMX?)

I think we should add new parameter to the all management APIs.

What do you think?

> 30 янв. 2020 г., в 12:51, Alexey Goncharuk <al...@gmail.com> написал(а):
> 
> Nikolay,
> 
> This change will affect only the command-line utility, correct? No changes
> for the public API are planned?
> 
> чт, 30 янв. 2020 г. в 12:46, Nikolay Izhikov <ni...@apache.org>:
> 
>> Hello, Alexey.
>> 
>> Thanks for the feedback.
>> It seems for me as a very unexpected behavior from the users point of view.
>> 
>> I propose to do the following:
>> 
>> * Update Ignite documentation and write down the fact that in-memory cache
>> cleared on deactivation.
>> * Disallow, by default, deactivation of the cluster that has in-memory
>> cache with proper error message
>>        «Your cluster has in-memory cache configured. During deactivation
>> all data from these caches will be cleared!»
>> * Add «—force» flag to deactivate command so administrator can force
>> deactivation for cluster that has both - persistent and in-memory caches
>> configured.
>> 
>> What do you think?
>> 
>> 
>>> 30 янв. 2020 г., в 12:32, Alexey Goncharuk <al...@gmail.com>
>> написал(а):
>>> 
>>> Nikolay,
>>> 
>>> Inactive state deallocates all possible resources by design, including
>> the
>>> data regions. If data region is not backed by a persistent storage, the
>>> data is lost, this is an expected behavior.
>>> 
>>> ср, 29 янв. 2020 г. в 19:18, Nikolay Izhikov <ni...@apache.org>:
>>> 
>>>> Hello, Igniters.
>>>> 
>>>> I found really confusing results of the simple test.
>>>> Data from the in-memory cache are vanished after change cluster state to
>>>> INACTIVE/ACTIVE.
>>>> 
>>>> Is this a bug or expected behavior?
>>>> 
>>>> 
>>>> ```
>>>> public class ClusterDeactivateTest extends GridCommonAbstractTest {
>>>>   String name = "my-cache";
>>>> 
>>>>   /** */
>>>>   @Test
>>>>   public void testDataPresent() throws Exception {
>>>>       IgniteEx i = startGrid(0);
>>>> 
>>>>       i.createCache(name).put(1L, 1L);
>>>> 
>>>>       assertEquals(1L, i.cache(name).get(1L));
>>>> 
>>>>       i.cluster().state(ClusterState.INACTIVE);
>>>>       i.cluster().state(ClusterState.ACTIVE);
>>>> 
>>>>       assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
>>>>   }
>>>> }
>>>> ```
>> 
>> 

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

Nikolay,

This change will affect only the command-line utility, correct? No changes
for the public API are planned?

чт, 30 янв. 2020 г. в 12:46, Nikolay Izhikov <ni...@apache.org>:

> Hello, Alexey.
>
> Thanks for the feedback.
> It seems for me as a very unexpected behavior from the users point of view.
>
> I propose to do the following:
>
> * Update Ignite documentation and write down the fact that in-memory cache
> cleared on deactivation.
> * Disallow, by default, deactivation of the cluster that has in-memory
> cache with proper error message
>         «Your cluster has in-memory cache configured. During deactivation
> all data from these caches will be cleared!»
> * Add «—force» flag to deactivate command so administrator can force
> deactivation for cluster that has both - persistent and in-memory caches
> configured.
>
> What do you think?
>
>
> > 30 янв. 2020 г., в 12:32, Alexey Goncharuk <al...@gmail.com>
> написал(а):
> >
> > Nikolay,
> >
> > Inactive state deallocates all possible resources by design, including
> the
> > data regions. If data region is not backed by a persistent storage, the
> > data is lost, this is an expected behavior.
> >
> > ср, 29 янв. 2020 г. в 19:18, Nikolay Izhikov <ni...@apache.org>:
> >
> >> Hello, Igniters.
> >>
> >> I found really confusing results of the simple test.
> >> Data from the in-memory cache are vanished after change cluster state to
> >> INACTIVE/ACTIVE.
> >>
> >> Is this a bug or expected behavior?
> >>
> >>
> >> ```
> >> public class ClusterDeactivateTest extends GridCommonAbstractTest {
> >>    String name = "my-cache";
> >>
> >>    /** */
> >>    @Test
> >>    public void testDataPresent() throws Exception {
> >>        IgniteEx i = startGrid(0);
> >>
> >>        i.createCache(name).put(1L, 1L);
> >>
> >>        assertEquals(1L, i.cache(name).get(1L));
> >>
> >>        i.cluster().state(ClusterState.INACTIVE);
> >>        i.cluster().state(ClusterState.ACTIVE);
> >>
> >>        assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
> >>    }
> >> }
> >> ```
>
>

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Nikolay Izhikov <ni...@apache.org>]

Hello, Alexey.

Thanks for the feedback.
It seems for me as a very unexpected behavior from the users point of view.

I propose to do the following:

* Update Ignite documentation and write down the fact that in-memory cache cleared on deactivation.
* Disallow, by default, deactivation of the cluster that has in-memory cache with proper error message
	«Your cluster has in-memory cache configured. During deactivation all data from these caches will be cleared!»
* Add «—force» flag to deactivate command so administrator can force deactivation for cluster that has both - persistent and in-memory caches configured.

What do you think?


> 30 янв. 2020 г., в 12:32, Alexey Goncharuk <al...@gmail.com> написал(а):
> 
> Nikolay,
> 
> Inactive state deallocates all possible resources by design, including the
> data regions. If data region is not backed by a persistent storage, the
> data is lost, this is an expected behavior.
> 
> ср, 29 янв. 2020 г. в 19:18, Nikolay Izhikov <ni...@apache.org>:
> 
>> Hello, Igniters.
>> 
>> I found really confusing results of the simple test.
>> Data from the in-memory cache are vanished after change cluster state to
>> INACTIVE/ACTIVE.
>> 
>> Is this a bug or expected behavior?
>> 
>> 
>> ```
>> public class ClusterDeactivateTest extends GridCommonAbstractTest {
>>    String name = "my-cache";
>> 
>>    /** */
>>    @Test
>>    public void testDataPresent() throws Exception {
>>        IgniteEx i = startGrid(0);
>> 
>>        i.createCache(name).put(1L, 1L);
>> 
>>        assertEquals(1L, i.cache(name).get(1L));
>> 
>>        i.cluster().state(ClusterState.INACTIVE);
>>        i.cluster().state(ClusterState.ACTIVE);
>> 
>>        assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
>>    }
>> }
>> ```

Re: Data vanished from cluster after INACTIVE/ACTIVE switch

[Alexey Goncharuk <al...@gmail.com>]

Nikolay,

Inactive state deallocates all possible resources by design, including the
data regions. If data region is not backed by a persistent storage, the
data is lost, this is an expected behavior.

ср, 29 янв. 2020 г. в 19:18, Nikolay Izhikov <ni...@apache.org>:

> Hello, Igniters.
>
> I found really confusing results of the simple test.
> Data from the in-memory cache are vanished after change cluster state to
> INACTIVE/ACTIVE.
>
> Is this a bug or expected behavior?
>
>
> ```
> public class ClusterDeactivateTest extends GridCommonAbstractTest {
>     String name = "my-cache";
>
>     /** */
>     @Test
>     public void testDataPresent() throws Exception {
>         IgniteEx i = startGrid(0);
>
>         i.createCache(name).put(1L, 1L);
>
>         assertEquals(1L, i.cache(name).get(1L));
>
>         i.cluster().state(ClusterState.INACTIVE);
>         i.cluster().state(ClusterState.ACTIVE);
>
>         assertEquals(1L, i.cache(name).get(1L)); //Assertion error here!
>     }
> }
> ```