You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ivan Rakov <iv...@gmail.com> on 2020/02/04 11:27:57 UTC
Forbid mixed cache groups with both atomic and transactional caches
Igniters,
Apparently it's possible in Ignite to configure a cache group with both
ATOMIC and TRANSACTIONAL caches.
Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
In my opinion, it would be better to remove such possibility from the
product. There are several reasons:
1) The original idea of grouping caches was optimizing storage overhead and
PME time by joining data of similar caches into the same partitions. ATOMIC
and TRANSACTIONAL caches provide different guarantees and are designed for
different use cases, thus they can hardly be called "similar".
2) Diving deeper: synchronization protocols and possible reasons for
primary-backup divergences are conceptually different for ATOMIC and
TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery protocol
allows to recover consistency if any participating node will fail, but for
ATOMIC caches there's possible scenario with failure of primary node where
neither of backups will contain the most recent state of the data. Example:
one backup have received updates 1, 3, 5 while another have received 2, 4
(which is possible due to message reordering), and even tracking counters
[1] won't restore the consistency. The problem is that we can't distinguish
what kind of conflict we have faced in case update counters have diverged
in a mixed group.
3) Mixed groups are poorly tested. I can't find any tests except a couple
of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
synchronization protocols will work correctly for such configurations,
especially under load and with a variety of dependent configuration
parameters.
4) I have never heard of any feedback on mixed groups. I have asked
different people on this and no one recalled any attempts to configure such
groups. I believe that in fact no one has ever tried to do it.
Please let me know if you are aware of any cases where mixed groups are
used or reasons to keep them. Otherwise I'll create a ticket to prohibit
mixed configurations.
[1]: https://issues.apache.org/jira/browse/IGNITE-11797
--
Best Regards,
Ivan Rakov
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Вячеслав Коптилин <sl...@gmail.com>.
Hello,
I absolutely agree with the proposed restriction. Additionally, this fact
should be clearly stated on the documentation page related to cache groups
https://apacheignite.readme.io/docs/cache-groups
Thanks,
S.
вт, 4 февр. 2020 г. в 14:40, Alexey Goncharuk <al...@gmail.com>:
> I support this change. While this has no much difference on the storage
> level, the protocols are indeed are very different and thus should be
> separated.
>
> вт, 4 февр. 2020 г. в 14:36, Anton Vinogradov <av...@apache.org>:
>
> > Seems, we already started the separation by atomic operations restriction
> > inside the transactions [1].
> > See no reason to allow mixes in this case.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-2313
> >
> > On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov <iv...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > Apparently it's possible in Ignite to configure a cache group with both
> > > ATOMIC and TRANSACTIONAL caches.
> > > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups*
> tests.
> > > In my opinion, it would be better to remove such possibility from the
> > > product. There are several reasons:
> > >
> > > 1) The original idea of grouping caches was optimizing storage overhead
> > and
> > > PME time by joining data of similar caches into the same partitions.
> > ATOMIC
> > > and TRANSACTIONAL caches provide different guarantees and are designed
> > for
> > > different use cases, thus they can hardly be called "similar".
> > >
> > > 2) Diving deeper: synchronization protocols and possible reasons for
> > > primary-backup divergences are conceptually different for ATOMIC and
> > > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> > protocol
> > > allows to recover consistency if any participating node will fail, but
> > for
> > > ATOMIC caches there's possible scenario with failure of primary node
> > where
> > > neither of backups will contain the most recent state of the data.
> > Example:
> > > one backup have received updates 1, 3, 5 while another have received
> 2, 4
> > > (which is possible due to message reordering), and even tracking
> counters
> > > [1] won't restore the consistency. The problem is that we can't
> > distinguish
> > > what kind of conflict we have faced in case update counters have
> diverged
> > > in a mixed group.
> > >
> > > 3) Mixed groups are poorly tested. I can't find any tests except a
> couple
> > > of smoke tests in IgniteCacheGroupsTest. We can't be sure that
> different
> > > synchronization protocols will work correctly for such configurations,
> > > especially under load and with a variety of dependent configuration
> > > parameters.
> > >
> > > 4) I have never heard of any feedback on mixed groups. I have asked
> > > different people on this and no one recalled any attempts to configure
> > such
> > > groups. I believe that in fact no one has ever tried to do it.
> > >
> > > Please let me know if you are aware of any cases where mixed groups are
> > > used or reasons to keep them. Otherwise I'll create a ticket to
> prohibit
> > > mixed configurations.
> > >
> > > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> > >
> > > --
> > > Best Regards,
> > > Ivan Rakov
> > >
> >
>
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Alexei Scherbakov <al...@gmail.com>.
+1 to avoid mixed tx-atomic cache groups.
On the other side, doing atomic ops inside a tx have no relation to
described reasons and is acceptable for me.
вт, 4 февр. 2020 г. в 14:40, Alexey Goncharuk <al...@gmail.com>:
> I support this change. While this has no much difference on the storage
> level, the protocols are indeed are very different and thus should be
> separated.
>
> вт, 4 февр. 2020 г. в 14:36, Anton Vinogradov <av...@apache.org>:
>
> > Seems, we already started the separation by atomic operations restriction
> > inside the transactions [1].
> > See no reason to allow mixes in this case.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-2313
> >
> > On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov <iv...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > Apparently it's possible in Ignite to configure a cache group with both
> > > ATOMIC and TRANSACTIONAL caches.
> > > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups*
> tests.
> > > In my opinion, it would be better to remove such possibility from the
> > > product. There are several reasons:
> > >
> > > 1) The original idea of grouping caches was optimizing storage overhead
> > and
> > > PME time by joining data of similar caches into the same partitions.
> > ATOMIC
> > > and TRANSACTIONAL caches provide different guarantees and are designed
> > for
> > > different use cases, thus they can hardly be called "similar".
> > >
> > > 2) Diving deeper: synchronization protocols and possible reasons for
> > > primary-backup divergences are conceptually different for ATOMIC and
> > > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> > protocol
> > > allows to recover consistency if any participating node will fail, but
> > for
> > > ATOMIC caches there's possible scenario with failure of primary node
> > where
> > > neither of backups will contain the most recent state of the data.
> > Example:
> > > one backup have received updates 1, 3, 5 while another have received
> 2, 4
> > > (which is possible due to message reordering), and even tracking
> counters
> > > [1] won't restore the consistency. The problem is that we can't
> > distinguish
> > > what kind of conflict we have faced in case update counters have
> diverged
> > > in a mixed group.
> > >
> > > 3) Mixed groups are poorly tested. I can't find any tests except a
> couple
> > > of smoke tests in IgniteCacheGroupsTest. We can't be sure that
> different
> > > synchronization protocols will work correctly for such configurations,
> > > especially under load and with a variety of dependent configuration
> > > parameters.
> > >
> > > 4) I have never heard of any feedback on mixed groups. I have asked
> > > different people on this and no one recalled any attempts to configure
> > such
> > > groups. I believe that in fact no one has ever tried to do it.
> > >
> > > Please let me know if you are aware of any cases where mixed groups are
> > > used or reasons to keep them. Otherwise I'll create a ticket to
> prohibit
> > > mixed configurations.
> > >
> > > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> > >
> > > --
> > > Best Regards,
> > > Ivan Rakov
> > >
> >
>
--
Best regards,
Alexei Scherbakov
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Alexey Goncharuk <al...@gmail.com>.
I support this change. While this has no much difference on the storage
level, the protocols are indeed are very different and thus should be
separated.
вт, 4 февр. 2020 г. в 14:36, Anton Vinogradov <av...@apache.org>:
> Seems, we already started the separation by atomic operations restriction
> inside the transactions [1].
> See no reason to allow mixes in this case.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-2313
>
> On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov <iv...@gmail.com> wrote:
>
> > Igniters,
> >
> > Apparently it's possible in Ignite to configure a cache group with both
> > ATOMIC and TRANSACTIONAL caches.
> > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> > In my opinion, it would be better to remove such possibility from the
> > product. There are several reasons:
> >
> > 1) The original idea of grouping caches was optimizing storage overhead
> and
> > PME time by joining data of similar caches into the same partitions.
> ATOMIC
> > and TRANSACTIONAL caches provide different guarantees and are designed
> for
> > different use cases, thus they can hardly be called "similar".
> >
> > 2) Diving deeper: synchronization protocols and possible reasons for
> > primary-backup divergences are conceptually different for ATOMIC and
> > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> protocol
> > allows to recover consistency if any participating node will fail, but
> for
> > ATOMIC caches there's possible scenario with failure of primary node
> where
> > neither of backups will contain the most recent state of the data.
> Example:
> > one backup have received updates 1, 3, 5 while another have received 2, 4
> > (which is possible due to message reordering), and even tracking counters
> > [1] won't restore the consistency. The problem is that we can't
> distinguish
> > what kind of conflict we have faced in case update counters have diverged
> > in a mixed group.
> >
> > 3) Mixed groups are poorly tested. I can't find any tests except a couple
> > of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> > synchronization protocols will work correctly for such configurations,
> > especially under load and with a variety of dependent configuration
> > parameters.
> >
> > 4) I have never heard of any feedback on mixed groups. I have asked
> > different people on this and no one recalled any attempts to configure
> such
> > groups. I believe that in fact no one has ever tried to do it.
> >
> > Please let me know if you are aware of any cases where mixed groups are
> > used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> > mixed configurations.
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
>
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Ivan Rakov <iv...@gmail.com>.
Anton,
Indeed, that's +1 point for forbidding mixed configurations.
On Tue, Feb 4, 2020 at 2:36 PM Anton Vinogradov <av...@apache.org> wrote:
> Seems, we already started the separation by atomic operations restriction
> inside the transactions [1].
> See no reason to allow mixes in this case.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-2313
>
> On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov <iv...@gmail.com> wrote:
>
> > Igniters,
> >
> > Apparently it's possible in Ignite to configure a cache group with both
> > ATOMIC and TRANSACTIONAL caches.
> > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> > In my opinion, it would be better to remove such possibility from the
> > product. There are several reasons:
> >
> > 1) The original idea of grouping caches was optimizing storage overhead
> and
> > PME time by joining data of similar caches into the same partitions.
> ATOMIC
> > and TRANSACTIONAL caches provide different guarantees and are designed
> for
> > different use cases, thus they can hardly be called "similar".
> >
> > 2) Diving deeper: synchronization protocols and possible reasons for
> > primary-backup divergences are conceptually different for ATOMIC and
> > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> protocol
> > allows to recover consistency if any participating node will fail, but
> for
> > ATOMIC caches there's possible scenario with failure of primary node
> where
> > neither of backups will contain the most recent state of the data.
> Example:
> > one backup have received updates 1, 3, 5 while another have received 2, 4
> > (which is possible due to message reordering), and even tracking counters
> > [1] won't restore the consistency. The problem is that we can't
> distinguish
> > what kind of conflict we have faced in case update counters have diverged
> > in a mixed group.
> >
> > 3) Mixed groups are poorly tested. I can't find any tests except a couple
> > of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> > synchronization protocols will work correctly for such configurations,
> > especially under load and with a variety of dependent configuration
> > parameters.
> >
> > 4) I have never heard of any feedback on mixed groups. I have asked
> > different people on this and no one recalled any attempts to configure
> such
> > groups. I believe that in fact no one has ever tried to do it.
> >
> > Please let me know if you are aware of any cases where mixed groups are
> > used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> > mixed configurations.
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
>
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Anton Vinogradov <av...@apache.org>.
Seems, we already started the separation by atomic operations restriction
inside the transactions [1].
See no reason to allow mixes in this case.
[1] https://issues.apache.org/jira/browse/IGNITE-2313
On Tue, Feb 4, 2020 at 2:28 PM Ivan Rakov <iv...@gmail.com> wrote:
> Igniters,
>
> Apparently it's possible in Ignite to configure a cache group with both
> ATOMIC and TRANSACTIONAL caches.
> Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> In my opinion, it would be better to remove such possibility from the
> product. There are several reasons:
>
> 1) The original idea of grouping caches was optimizing storage overhead and
> PME time by joining data of similar caches into the same partitions. ATOMIC
> and TRANSACTIONAL caches provide different guarantees and are designed for
> different use cases, thus they can hardly be called "similar".
>
> 2) Diving deeper: synchronization protocols and possible reasons for
> primary-backup divergences are conceptually different for ATOMIC and
> TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery protocol
> allows to recover consistency if any participating node will fail, but for
> ATOMIC caches there's possible scenario with failure of primary node where
> neither of backups will contain the most recent state of the data. Example:
> one backup have received updates 1, 3, 5 while another have received 2, 4
> (which is possible due to message reordering), and even tracking counters
> [1] won't restore the consistency. The problem is that we can't distinguish
> what kind of conflict we have faced in case update counters have diverged
> in a mixed group.
>
> 3) Mixed groups are poorly tested. I can't find any tests except a couple
> of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> synchronization protocols will work correctly for such configurations,
> especially under load and with a variety of dependent configuration
> parameters.
>
> 4) I have never heard of any feedback on mixed groups. I have asked
> different people on this and no one recalled any attempts to configure such
> groups. I believe that in fact no one has ever tried to do it.
>
> Please let me know if you are aware of any cases where mixed groups are
> used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> mixed configurations.
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-11797
>
> --
> Best Regards,
> Ivan Rakov
>
Re: Forbid mixed cache groups with both atomic and transactional
caches
Posted by Nikolay Izhikov <ni...@apache.org>.
Ivan.
It seems we don’t have a format definition for a «community decision»
We, for sure, should fill this gap.
Me and Andrey Gura, have certain proposals for our development process based on metrics API discussion.
We will share those proposals after 2.8 release and some additional discussion.
> 5 февр. 2020 г., в 13:11, Ivan Pavlukhin <vo...@gmail.com> написал(а):
>
> Folks,
>
> A bit of offtop. Do we have some recommendations in the community how
> long should we wait until treating something as "a Community
> conclusion"? It worries me a little bit that I see a discussion for a
> first time and there is already a conclusion. And the discussion was
> started lesser than 24 hours ago. I suppose we should allow everyone
> interested to share an opinion (here I agree with the proposal) and it
> usually requires some time in open-source communities.
>
> ср, 5 февр. 2020 г. в 10:58, Ivan Rakov <iv...@gmail.com>:
>>
>> Folks,
>>
>> Thanks for your feedback.
>> I've created a JIRA issue on this change:
>> https://issues.apache.org/jira/browse/IGNITE-12622
>>
>> On Tue, Feb 4, 2020 at 10:43 PM Denis Magda <dm...@apache.org> wrote:
>>
>>> +1 from my end. It doesn't sound like a big deal if Ignite users need to
>>> define separate groups for atomic and transactional caches.
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>>> Igniters,
>>>>
>>>> Apparently it's possible in Ignite to configure a cache group with both
>>>> ATOMIC and TRANSACTIONAL caches.
>>>> Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
>>>> In my opinion, it would be better to remove such possibility from the
>>>> product. There are several reasons:
>>>>
>>>> 1) The original idea of grouping caches was optimizing storage overhead
>>> and
>>>> PME time by joining data of similar caches into the same partitions.
>>> ATOMIC
>>>> and TRANSACTIONAL caches provide different guarantees and are designed
>>> for
>>>> different use cases, thus they can hardly be called "similar".
>>>>
>>>> 2) Diving deeper: synchronization protocols and possible reasons for
>>>> primary-backup divergences are conceptually different for ATOMIC and
>>>> TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
>>> protocol
>>>> allows to recover consistency if any participating node will fail, but
>>> for
>>>> ATOMIC caches there's possible scenario with failure of primary node
>>> where
>>>> neither of backups will contain the most recent state of the data.
>>> Example:
>>>> one backup have received updates 1, 3, 5 while another have received 2, 4
>>>> (which is possible due to message reordering), and even tracking counters
>>>> [1] won't restore the consistency. The problem is that we can't
>>> distinguish
>>>> what kind of conflict we have faced in case update counters have diverged
>>>> in a mixed group.
>>>>
>>>> 3) Mixed groups are poorly tested. I can't find any tests except a couple
>>>> of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
>>>> synchronization protocols will work correctly for such configurations,
>>>> especially under load and with a variety of dependent configuration
>>>> parameters.
>>>>
>>>> 4) I have never heard of any feedback on mixed groups. I have asked
>>>> different people on this and no one recalled any attempts to configure
>>> such
>>>> groups. I believe that in fact no one has ever tried to do it.
>>>>
>>>> Please let me know if you are aware of any cases where mixed groups are
>>>> used or reasons to keep them. Otherwise I'll create a ticket to prohibit
>>>> mixed configurations.
>>>>
>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-11797
>>>>
>>>> --
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Nikolay, Ivan,
Thank you guys! I knew that I should not worry =)
Best regards,
Ivan Pavlukhin
ср, 5 февр. 2020 г. в 13:46, Ivan Rakov <iv...@gmail.com>:
>
> Ivan,
>
> Thanks for pointing this out. Less than one day is indeed too early to
> treat this discussion thread as a "community conclusion". Still, the
> consensus among the current participants made me feel that a conclusion
> will be reached.
> We'll surely get back to the discussion if opposite opinions will arise.
>
> On Wed, Feb 5, 2020 at 1:11 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
>
> > Folks,
> >
> > A bit of offtop. Do we have some recommendations in the community how
> > long should we wait until treating something as "a Community
> > conclusion"? It worries me a little bit that I see a discussion for a
> > first time and there is already a conclusion. And the discussion was
> > started lesser than 24 hours ago. I suppose we should allow everyone
> > interested to share an opinion (here I agree with the proposal) and it
> > usually requires some time in open-source communities.
> >
> > ср, 5 февр. 2020 г. в 10:58, Ivan Rakov <iv...@gmail.com>:
> > >
> > > Folks,
> > >
> > > Thanks for your feedback.
> > > I've created a JIRA issue on this change:
> > > https://issues.apache.org/jira/browse/IGNITE-12622
> > >
> > > On Tue, Feb 4, 2020 at 10:43 PM Denis Magda <dm...@apache.org> wrote:
> > >
> > > > +1 from my end. It doesn't sound like a big deal if Ignite users need
> > to
> > > > define separate groups for atomic and transactional caches.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com>
> > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Apparently it's possible in Ignite to configure a cache group with
> > both
> > > > > ATOMIC and TRANSACTIONAL caches.
> > > > > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups*
> > tests.
> > > > > In my opinion, it would be better to remove such possibility from the
> > > > > product. There are several reasons:
> > > > >
> > > > > 1) The original idea of grouping caches was optimizing storage
> > overhead
> > > > and
> > > > > PME time by joining data of similar caches into the same partitions.
> > > > ATOMIC
> > > > > and TRANSACTIONAL caches provide different guarantees and are
> > designed
> > > > for
> > > > > different use cases, thus they can hardly be called "similar".
> > > > >
> > > > > 2) Diving deeper: synchronization protocols and possible reasons for
> > > > > primary-backup divergences are conceptually different for ATOMIC and
> > > > > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> > > > protocol
> > > > > allows to recover consistency if any participating node will fail,
> > but
> > > > for
> > > > > ATOMIC caches there's possible scenario with failure of primary node
> > > > where
> > > > > neither of backups will contain the most recent state of the data.
> > > > Example:
> > > > > one backup have received updates 1, 3, 5 while another have received
> > 2, 4
> > > > > (which is possible due to message reordering), and even tracking
> > counters
> > > > > [1] won't restore the consistency. The problem is that we can't
> > > > distinguish
> > > > > what kind of conflict we have faced in case update counters have
> > diverged
> > > > > in a mixed group.
> > > > >
> > > > > 3) Mixed groups are poorly tested. I can't find any tests except a
> > couple
> > > > > of smoke tests in IgniteCacheGroupsTest. We can't be sure that
> > different
> > > > > synchronization protocols will work correctly for such
> > configurations,
> > > > > especially under load and with a variety of dependent configuration
> > > > > parameters.
> > > > >
> > > > > 4) I have never heard of any feedback on mixed groups. I have asked
> > > > > different people on this and no one recalled any attempts to
> > configure
> > > > such
> > > > > groups. I believe that in fact no one has ever tried to do it.
> > > > >
> > > > > Please let me know if you are aware of any cases where mixed groups
> > are
> > > > > used or reasons to keep them. Otherwise I'll create a ticket to
> > prohibit
> > > > > mixed configurations.
> > > > >
> > > > > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > > Ivan Rakov
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Ivan Rakov <iv...@gmail.com>.
Ivan,
Thanks for pointing this out. Less than one day is indeed too early to
treat this discussion thread as a "community conclusion". Still, the
consensus among the current participants made me feel that a conclusion
will be reached.
We'll surely get back to the discussion if opposite opinions will arise.
On Wed, Feb 5, 2020 at 1:11 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Folks,
>
> A bit of offtop. Do we have some recommendations in the community how
> long should we wait until treating something as "a Community
> conclusion"? It worries me a little bit that I see a discussion for a
> first time and there is already a conclusion. And the discussion was
> started lesser than 24 hours ago. I suppose we should allow everyone
> interested to share an opinion (here I agree with the proposal) and it
> usually requires some time in open-source communities.
>
> ср, 5 февр. 2020 г. в 10:58, Ivan Rakov <iv...@gmail.com>:
> >
> > Folks,
> >
> > Thanks for your feedback.
> > I've created a JIRA issue on this change:
> > https://issues.apache.org/jira/browse/IGNITE-12622
> >
> > On Tue, Feb 4, 2020 at 10:43 PM Denis Magda <dm...@apache.org> wrote:
> >
> > > +1 from my end. It doesn't sound like a big deal if Ignite users need
> to
> > > define separate groups for atomic and transactional caches.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com>
> wrote:
> > >
> > > > Igniters,
> > > >
> > > > Apparently it's possible in Ignite to configure a cache group with
> both
> > > > ATOMIC and TRANSACTIONAL caches.
> > > > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups*
> tests.
> > > > In my opinion, it would be better to remove such possibility from the
> > > > product. There are several reasons:
> > > >
> > > > 1) The original idea of grouping caches was optimizing storage
> overhead
> > > and
> > > > PME time by joining data of similar caches into the same partitions.
> > > ATOMIC
> > > > and TRANSACTIONAL caches provide different guarantees and are
> designed
> > > for
> > > > different use cases, thus they can hardly be called "similar".
> > > >
> > > > 2) Diving deeper: synchronization protocols and possible reasons for
> > > > primary-backup divergences are conceptually different for ATOMIC and
> > > > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> > > protocol
> > > > allows to recover consistency if any participating node will fail,
> but
> > > for
> > > > ATOMIC caches there's possible scenario with failure of primary node
> > > where
> > > > neither of backups will contain the most recent state of the data.
> > > Example:
> > > > one backup have received updates 1, 3, 5 while another have received
> 2, 4
> > > > (which is possible due to message reordering), and even tracking
> counters
> > > > [1] won't restore the consistency. The problem is that we can't
> > > distinguish
> > > > what kind of conflict we have faced in case update counters have
> diverged
> > > > in a mixed group.
> > > >
> > > > 3) Mixed groups are poorly tested. I can't find any tests except a
> couple
> > > > of smoke tests in IgniteCacheGroupsTest. We can't be sure that
> different
> > > > synchronization protocols will work correctly for such
> configurations,
> > > > especially under load and with a variety of dependent configuration
> > > > parameters.
> > > >
> > > > 4) I have never heard of any feedback on mixed groups. I have asked
> > > > different people on this and no one recalled any attempts to
> configure
> > > such
> > > > groups. I believe that in fact no one has ever tried to do it.
> > > >
> > > > Please let me know if you are aware of any cases where mixed groups
> are
> > > > used or reasons to keep them. Otherwise I'll create a ticket to
> prohibit
> > > > mixed configurations.
> > > >
> > > > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> > > >
> > > > --
> > > > Best Regards,
> > > > Ivan Rakov
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Folks,
A bit of offtop. Do we have some recommendations in the community how
long should we wait until treating something as "a Community
conclusion"? It worries me a little bit that I see a discussion for a
first time and there is already a conclusion. And the discussion was
started lesser than 24 hours ago. I suppose we should allow everyone
interested to share an opinion (here I agree with the proposal) and it
usually requires some time in open-source communities.
ср, 5 февр. 2020 г. в 10:58, Ivan Rakov <iv...@gmail.com>:
>
> Folks,
>
> Thanks for your feedback.
> I've created a JIRA issue on this change:
> https://issues.apache.org/jira/browse/IGNITE-12622
>
> On Tue, Feb 4, 2020 at 10:43 PM Denis Magda <dm...@apache.org> wrote:
>
> > +1 from my end. It doesn't sound like a big deal if Ignite users need to
> > define separate groups for atomic and transactional caches.
> >
> > -
> > Denis
> >
> >
> > On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > Apparently it's possible in Ignite to configure a cache group with both
> > > ATOMIC and TRANSACTIONAL caches.
> > > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> > > In my opinion, it would be better to remove such possibility from the
> > > product. There are several reasons:
> > >
> > > 1) The original idea of grouping caches was optimizing storage overhead
> > and
> > > PME time by joining data of similar caches into the same partitions.
> > ATOMIC
> > > and TRANSACTIONAL caches provide different guarantees and are designed
> > for
> > > different use cases, thus they can hardly be called "similar".
> > >
> > > 2) Diving deeper: synchronization protocols and possible reasons for
> > > primary-backup divergences are conceptually different for ATOMIC and
> > > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> > protocol
> > > allows to recover consistency if any participating node will fail, but
> > for
> > > ATOMIC caches there's possible scenario with failure of primary node
> > where
> > > neither of backups will contain the most recent state of the data.
> > Example:
> > > one backup have received updates 1, 3, 5 while another have received 2, 4
> > > (which is possible due to message reordering), and even tracking counters
> > > [1] won't restore the consistency. The problem is that we can't
> > distinguish
> > > what kind of conflict we have faced in case update counters have diverged
> > > in a mixed group.
> > >
> > > 3) Mixed groups are poorly tested. I can't find any tests except a couple
> > > of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> > > synchronization protocols will work correctly for such configurations,
> > > especially under load and with a variety of dependent configuration
> > > parameters.
> > >
> > > 4) I have never heard of any feedback on mixed groups. I have asked
> > > different people on this and no one recalled any attempts to configure
> > such
> > > groups. I believe that in fact no one has ever tried to do it.
> > >
> > > Please let me know if you are aware of any cases where mixed groups are
> > > used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> > > mixed configurations.
> > >
> > > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> > >
> > > --
> > > Best Regards,
> > > Ivan Rakov
> > >
> >
--
Best regards,
Ivan Pavlukhin
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Ivan Rakov <iv...@gmail.com>.
Folks,
Thanks for your feedback.
I've created a JIRA issue on this change:
https://issues.apache.org/jira/browse/IGNITE-12622
On Tue, Feb 4, 2020 at 10:43 PM Denis Magda <dm...@apache.org> wrote:
> +1 from my end. It doesn't sound like a big deal if Ignite users need to
> define separate groups for atomic and transactional caches.
>
> -
> Denis
>
>
> On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com> wrote:
>
> > Igniters,
> >
> > Apparently it's possible in Ignite to configure a cache group with both
> > ATOMIC and TRANSACTIONAL caches.
> > Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> > In my opinion, it would be better to remove such possibility from the
> > product. There are several reasons:
> >
> > 1) The original idea of grouping caches was optimizing storage overhead
> and
> > PME time by joining data of similar caches into the same partitions.
> ATOMIC
> > and TRANSACTIONAL caches provide different guarantees and are designed
> for
> > different use cases, thus they can hardly be called "similar".
> >
> > 2) Diving deeper: synchronization protocols and possible reasons for
> > primary-backup divergences are conceptually different for ATOMIC and
> > TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery
> protocol
> > allows to recover consistency if any participating node will fail, but
> for
> > ATOMIC caches there's possible scenario with failure of primary node
> where
> > neither of backups will contain the most recent state of the data.
> Example:
> > one backup have received updates 1, 3, 5 while another have received 2, 4
> > (which is possible due to message reordering), and even tracking counters
> > [1] won't restore the consistency. The problem is that we can't
> distinguish
> > what kind of conflict we have faced in case update counters have diverged
> > in a mixed group.
> >
> > 3) Mixed groups are poorly tested. I can't find any tests except a couple
> > of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> > synchronization protocols will work correctly for such configurations,
> > especially under load and with a variety of dependent configuration
> > parameters.
> >
> > 4) I have never heard of any feedback on mixed groups. I have asked
> > different people on this and no one recalled any attempts to configure
> such
> > groups. I believe that in fact no one has ever tried to do it.
> >
> > Please let me know if you are aware of any cases where mixed groups are
> > used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> > mixed configurations.
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-11797
> >
> > --
> > Best Regards,
> > Ivan Rakov
> >
>
Re: Forbid mixed cache groups with both atomic and transactional caches
Posted by Denis Magda <dm...@apache.org>.
+1 from my end. It doesn't sound like a big deal if Ignite users need to
define separate groups for atomic and transactional caches.
-
Denis
On Tue, Feb 4, 2020 at 3:28 AM Ivan Rakov <iv...@gmail.com> wrote:
> Igniters,
>
> Apparently it's possible in Ignite to configure a cache group with both
> ATOMIC and TRANSACTIONAL caches.
> Proof: IgniteCacheGroupsTest#testContinuousQueriesMultipleGroups* tests.
> In my opinion, it would be better to remove such possibility from the
> product. There are several reasons:
>
> 1) The original idea of grouping caches was optimizing storage overhead and
> PME time by joining data of similar caches into the same partitions. ATOMIC
> and TRANSACTIONAL caches provide different guarantees and are designed for
> different use cases, thus they can hardly be called "similar".
>
> 2) Diving deeper: synchronization protocols and possible reasons for
> primary-backup divergences are conceptually different for ATOMIC and
> TRANSACTIONAL cases. In TRANSACTIONAL case, transactions recovery protocol
> allows to recover consistency if any participating node will fail, but for
> ATOMIC caches there's possible scenario with failure of primary node where
> neither of backups will contain the most recent state of the data. Example:
> one backup have received updates 1, 3, 5 while another have received 2, 4
> (which is possible due to message reordering), and even tracking counters
> [1] won't restore the consistency. The problem is that we can't distinguish
> what kind of conflict we have faced in case update counters have diverged
> in a mixed group.
>
> 3) Mixed groups are poorly tested. I can't find any tests except a couple
> of smoke tests in IgniteCacheGroupsTest. We can't be sure that different
> synchronization protocols will work correctly for such configurations,
> especially under load and with a variety of dependent configuration
> parameters.
>
> 4) I have never heard of any feedback on mixed groups. I have asked
> different people on this and no one recalled any attempts to configure such
> groups. I believe that in fact no one has ever tried to do it.
>
> Please let me know if you are aware of any cases where mixed groups are
> used or reasons to keep them. Otherwise I'll create a ticket to prohibit
> mixed configurations.
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-11797
>
> --
> Best Regards,
> Ivan Rakov
>