You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Vladimir Ershov <ve...@gridgain.com> on 2016/01/13 14:23:19 UTC
New method in IgniteCache API.
Igniters!
Since we are going to add stronger data loss check IGNITE-1605
<https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add new
method for the manual resetting state of the lost partition in IgniteCache
IgniteFuture<?> *resetLostPartitions*();
just near #rebalance().
And would like to ask the community: Does *resetLostPartitions *naming
sound good?
Thoughts appreciated,
Thanks!
Re: New method in IgniteCache API.
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I still believe the difference is very subtle for a naked eye. Seems like
both, topology validator and data loss check are almost identical:
- both get invoked whenever topology changes
- both can prohibit some operation on cache, e.g. updates in case of
topology validator or updates/reads/both in case of data loss check
I think the design screams for these two abstractions to be combined into
one. Otherwise, we are only going to muddle the API.
D.
On Fri, Jan 15, 2016 at 12:17 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Still not sure how they are similar:
>
> TopologyValidator is invoked on every topology change and provides a
> boolean result whether topology is ready to be used. Currently there is no
> way to mark a 'failed' topology as valid without another topology change.
>
> This new method that Vladimir suggests resets the partition state after it
> has been lost, it does not invoke topology validator and does not need a
> topology change to work.
>
> 2016-01-15 10:59 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>
> > Alexey,
> >
> > All I am saying is that the difference between topology validator and
> this
> > new method becomes very subtle. One just looks like a subset of another.
> Am
> > I wrong?
> >
> > D.
> >
> > On Thu, Jan 14, 2016 at 2:38 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Dmitriy,
> > >
> > > Are you suggesting that we need to pass partitions state to a topology
> > > validator so that user needs to check it manually. I do not think this
> is
> > > convenient for an end-user and like the approach with the policy that
> > > Vladimir suggested better.
> > >
> > > Raul,
> > >
> > > I assume you want to add IgniteCacheEx interface? How one would get it?
> > > Another thing that crossed my mind was that it may be more efficient to
> > > reset the state of multiple caches simultaneously, so maybe we should
> add
> > > this method to Ignite and pass a list of cache names?
> > >
> >
>
Re: New method in IgniteCache API.
Posted by Alexey Goncharuk <al...@gmail.com>.
Still not sure how they are similar:
TopologyValidator is invoked on every topology change and provides a
boolean result whether topology is ready to be used. Currently there is no
way to mark a 'failed' topology as valid without another topology change.
This new method that Vladimir suggests resets the partition state after it
has been lost, it does not invoke topology validator and does not need a
topology change to work.
2016-01-15 10:59 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> Alexey,
>
> All I am saying is that the difference between topology validator and this
> new method becomes very subtle. One just looks like a subset of another. Am
> I wrong?
>
> D.
>
> On Thu, Jan 14, 2016 at 2:38 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Dmitriy,
> >
> > Are you suggesting that we need to pass partitions state to a topology
> > validator so that user needs to check it manually. I do not think this is
> > convenient for an end-user and like the approach with the policy that
> > Vladimir suggested better.
> >
> > Raul,
> >
> > I assume you want to add IgniteCacheEx interface? How one would get it?
> > Another thing that crossed my mind was that it may be more efficient to
> > reset the state of multiple caches simultaneously, so maybe we should add
> > this method to Ignite and pass a list of cache names?
> >
>
Re: New method in IgniteCache API.
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Alexey,
All I am saying is that the difference between topology validator and this
new method becomes very subtle. One just looks like a subset of another. Am
I wrong?
D.
On Thu, Jan 14, 2016 at 2:38 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Dmitriy,
>
> Are you suggesting that we need to pass partitions state to a topology
> validator so that user needs to check it manually. I do not think this is
> convenient for an end-user and like the approach with the policy that
> Vladimir suggested better.
>
> Raul,
>
> I assume you want to add IgniteCacheEx interface? How one would get it?
> Another thing that crossed my mind was that it may be more efficient to
> reset the state of multiple caches simultaneously, so maybe we should add
> this method to Ignite and pass a list of cache names?
>
Re: New method in IgniteCache API.
Posted by Vladimir Ershov <ve...@gridgain.com>.
Ok, I've added our thoughts from this thread to the second stage of this
task:
1. IGNITE-2378 <https://issues.apache.org/jira/browse/IGNITE-2378> Data
loss Part 2. Add flexible DataLoss Policy. Improve dataLoss event with
batch.
Feel free to comment it with all new ideas you come across.
As for now, I assume supporting possibility for batching of reset request
to be a great point, I'm going to implement this way.
Thanks everyone!
On Thu, Jan 14, 2016 at 1:38 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Dmitriy,
>
> Are you suggesting that we need to pass partitions state to a topology
> validator so that user needs to check it manually. I do not think this is
> convenient for an end-user and like the approach with the policy that
> Vladimir suggested better.
>
> Raul,
>
> I assume you want to add IgniteCacheEx interface? How one would get it?
> Another thing that crossed my mind was that it may be more efficient to
> reset the state of multiple caches simultaneously, so maybe we should add
> this method to Ignite and pass a list of cache names?
>
Re: New method in IgniteCache API.
Posted by Alexey Goncharuk <al...@gmail.com>.
Dmitriy,
Are you suggesting that we need to pass partitions state to a topology
validator so that user needs to check it manually. I do not think this is
convenient for an end-user and like the approach with the policy that
Vladimir suggested better.
Raul,
I assume you want to add IgniteCacheEx interface? How one would get it?
Another thing that crossed my mind was that it may be more efficient to
reset the state of multiple caches simultaneously, so maybe we should add
this method to Ignite and pass a list of cache names?
Re: New method in IgniteCache API.
Posted by Raul Kripalani <ra...@apache.org>.
Hello,
How often will end users invoke this functionality? It sounds somewhat like
an administrative function.
If it won't be invoked frequently, and its function is not on par with
get(), put(), etc., I would propose to add the method to IgniteCacheEx
instead, to avoid excessively polluting IgniteCache.
Raúl.
On 13 Jan 2016 13:23, "Vladimir Ershov" <ve...@gridgain.com> wrote:
> Igniters!
>
> Since we are going to add stronger data loss check IGNITE-1605
> <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add new
> method for the manual resetting state of the lost partition in IgniteCache
> IgniteFuture<?> *resetLostPartitions*();
> just near #rebalance().
>
> And would like to ask the community: Does *resetLostPartitions *naming
> sound good?
> Thoughts appreciated,
> Thanks!
>
Re: New method in IgniteCache API.
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am still not sure why this is not a Topology Validator with more context.
Why do we need to add yet another concept here?
On Wed, Jan 13, 2016 at 12:28 PM, Vladimir Ershov <ve...@gridgain.com>
wrote:
> Of course!
>
> 1. Yes, it is a part of the task. Documentation is about to be placed at
> readmeIo under 1.6 version.
> 2. No, but both those calls are near in the
> GridDhtTopologyExchangeFuture#validateCache.
> 3. Yes, sure. As far as I understand, main purpose of the IGNITE-1605
> <https://issues.apache.org/jira/browse/IGNITE-1605> is to implement
> necessary infrastructure for handling lost partitions and thus only first
> to edge DataLossPolicy is planned: FAIL_OPS and NOOP. Nonetheless, we a
> going to add more different DataLossPolicy during next tickets, but I'm not
> aware of the release version for them. It could be 1.6 or later.
> <https://issues.apache.org/jira/browse/IGNITE-1605>
>
> On Wed, Jan 13, 2016 at 9:56 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Thanks Vladimir!
> >
> > I have a few questions:
> >
> > 1. Is this being documented?
> > 2. Is this part of the topology validator?
> > 3. Are we going to block reads and writes or only writes? Ideally, user
> > should be able to configure either.
> >
> > D.
> >
> > On Wed, Jan 13, 2016 at 9:05 AM, Vladimir Ershov <ve...@gridgain.com>
> > wrote:
> >
> > > Dmitriy,
> > > I've added new GridDhtPartitionState.LOST state for partitions that
> were
> > > lost, and also new CacheConfiguration property - DataLossPolicy. If
> > > DataLossPolicy is FAIL_OPS, then all lost partitions get LOST state,
> and
> > > all actions for any keys on those lost partitions throw exception,
> until
> > > lost partition state is not reset by *resetLostPartitions*.
> > >
> > > On Wed, Jan 13, 2016 at 7:42 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > Can you please describe what this method will be doing? I am note
> sure
> > I
> > > > understand what resetting state of the lost partition means.
> > > >
> > > > D.
> > > >
> > > > On Wed, Jan 13, 2016 at 5:23 AM, Vladimir Ershov <
> vershov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Igniters!
> > > > >
> > > > > Since we are going to add stronger data loss check IGNITE-1605
> > > > > <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to
> add
> > > new
> > > > > method for the manual resetting state of the lost partition in
> > > > IgniteCache
> > > > > IgniteFuture<?> *resetLostPartitions*();
> > > > > just near #rebalance().
> > > > >
> > > > > And would like to ask the community: Does *resetLostPartitions
> > *naming
> > > > > sound good?
> > > > > Thoughts appreciated,
> > > > > Thanks!
> > > > >
> > > >
> > >
> >
>
Re: New method in IgniteCache API.
Posted by Vladimir Ershov <ve...@gridgain.com>.
Of course!
1. Yes, it is a part of the task. Documentation is about to be placed at
readmeIo under 1.6 version.
2. No, but both those calls are near in the
GridDhtTopologyExchangeFuture#validateCache.
3. Yes, sure. As far as I understand, main purpose of the IGNITE-1605
<https://issues.apache.org/jira/browse/IGNITE-1605> is to implement
necessary infrastructure for handling lost partitions and thus only first
to edge DataLossPolicy is planned: FAIL_OPS and NOOP. Nonetheless, we a
going to add more different DataLossPolicy during next tickets, but I'm not
aware of the release version for them. It could be 1.6 or later.
<https://issues.apache.org/jira/browse/IGNITE-1605>
On Wed, Jan 13, 2016 at 9:56 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Thanks Vladimir!
>
> I have a few questions:
>
> 1. Is this being documented?
> 2. Is this part of the topology validator?
> 3. Are we going to block reads and writes or only writes? Ideally, user
> should be able to configure either.
>
> D.
>
> On Wed, Jan 13, 2016 at 9:05 AM, Vladimir Ershov <ve...@gridgain.com>
> wrote:
>
> > Dmitriy,
> > I've added new GridDhtPartitionState.LOST state for partitions that were
> > lost, and also new CacheConfiguration property - DataLossPolicy. If
> > DataLossPolicy is FAIL_OPS, then all lost partitions get LOST state, and
> > all actions for any keys on those lost partitions throw exception, until
> > lost partition state is not reset by *resetLostPartitions*.
> >
> > On Wed, Jan 13, 2016 at 7:42 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > Vladimir,
> > >
> > > Can you please describe what this method will be doing? I am note sure
> I
> > > understand what resetting state of the lost partition means.
> > >
> > > D.
> > >
> > > On Wed, Jan 13, 2016 at 5:23 AM, Vladimir Ershov <vershov@gridgain.com
> >
> > > wrote:
> > >
> > > > Igniters!
> > > >
> > > > Since we are going to add stronger data loss check IGNITE-1605
> > > > <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add
> > new
> > > > method for the manual resetting state of the lost partition in
> > > IgniteCache
> > > > IgniteFuture<?> *resetLostPartitions*();
> > > > just near #rebalance().
> > > >
> > > > And would like to ask the community: Does *resetLostPartitions
> *naming
> > > > sound good?
> > > > Thoughts appreciated,
> > > > Thanks!
> > > >
> > >
> >
>
Re: New method in IgniteCache API.
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Thanks Vladimir!
I have a few questions:
1. Is this being documented?
2. Is this part of the topology validator?
3. Are we going to block reads and writes or only writes? Ideally, user
should be able to configure either.
D.
On Wed, Jan 13, 2016 at 9:05 AM, Vladimir Ershov <ve...@gridgain.com>
wrote:
> Dmitriy,
> I've added new GridDhtPartitionState.LOST state for partitions that were
> lost, and also new CacheConfiguration property - DataLossPolicy. If
> DataLossPolicy is FAIL_OPS, then all lost partitions get LOST state, and
> all actions for any keys on those lost partitions throw exception, until
> lost partition state is not reset by *resetLostPartitions*.
>
> On Wed, Jan 13, 2016 at 7:42 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Vladimir,
> >
> > Can you please describe what this method will be doing? I am note sure I
> > understand what resetting state of the lost partition means.
> >
> > D.
> >
> > On Wed, Jan 13, 2016 at 5:23 AM, Vladimir Ershov <ve...@gridgain.com>
> > wrote:
> >
> > > Igniters!
> > >
> > > Since we are going to add stronger data loss check IGNITE-1605
> > > <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add
> new
> > > method for the manual resetting state of the lost partition in
> > IgniteCache
> > > IgniteFuture<?> *resetLostPartitions*();
> > > just near #rebalance().
> > >
> > > And would like to ask the community: Does *resetLostPartitions *naming
> > > sound good?
> > > Thoughts appreciated,
> > > Thanks!
> > >
> >
>
Re: New method in IgniteCache API.
Posted by Vladimir Ershov <ve...@gridgain.com>.
Dmitriy,
I've added new GridDhtPartitionState.LOST state for partitions that were
lost, and also new CacheConfiguration property - DataLossPolicy. If
DataLossPolicy is FAIL_OPS, then all lost partitions get LOST state, and
all actions for any keys on those lost partitions throw exception, until
lost partition state is not reset by *resetLostPartitions*.
On Wed, Jan 13, 2016 at 7:42 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Vladimir,
>
> Can you please describe what this method will be doing? I am note sure I
> understand what resetting state of the lost partition means.
>
> D.
>
> On Wed, Jan 13, 2016 at 5:23 AM, Vladimir Ershov <ve...@gridgain.com>
> wrote:
>
> > Igniters!
> >
> > Since we are going to add stronger data loss check IGNITE-1605
> > <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add new
> > method for the manual resetting state of the lost partition in
> IgniteCache
> > IgniteFuture<?> *resetLostPartitions*();
> > just near #rebalance().
> >
> > And would like to ask the community: Does *resetLostPartitions *naming
> > sound good?
> > Thoughts appreciated,
> > Thanks!
> >
>
Re: New method in IgniteCache API.
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Vladimir,
Can you please describe what this method will be doing? I am note sure I
understand what resetting state of the lost partition means.
D.
On Wed, Jan 13, 2016 at 5:23 AM, Vladimir Ershov <ve...@gridgain.com>
wrote:
> Igniters!
>
> Since we are going to add stronger data loss check IGNITE-1605
> <https://issues.apache.org/jira/browse/IGNITE-1605> I'm about to add new
> method for the manual resetting state of the lost partition in IgniteCache
> IgniteFuture<?> *resetLostPartitions*();
> just near #rebalance().
>
> And would like to ask the community: Does *resetLostPartitions *naming
> sound good?
> Thoughts appreciated,
> Thanks!
>