You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Valentin Kulichenko <va...@gmail.com> on 2017/01/18 20:16:31 UTC
Partition data lost event
Folks,
Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is fired?
I just create a small test and the result confused me. This is what I do:
1. Start several server nodes, all listen to
EVT_CACHE_REBALANCE_PART_DATA_LOST event.
2. Create a cache with no backups.
3. Put some values.
4. Stop one of the nodes (lose some partitions).
What I noticed is that the event is fired on all nodes that are still in
topology, but for different partitions. For example, if there are 4
partitions lost (1,2,3,4), I can get notifications for 1,3 on first node
and 2,4 on second node. This seems weird and it's not clear to me how to
use this event.
Can anyone explain this behavior? Is this expected?
Here is the test code:
public class PartitionTest {
public static void main(String[] args) throws InterruptedException {
for (int i = 0; i < 4; i++)
Ignition.start(config("server-" + i));
Ignition.setClientMode(true);
Ignite ignite = Ignition.start();
CacheConfiguration<Integer, Integer> cfg = new
CacheConfiguration<>("test");
cfg.setAffinity(new RendezvousAffinityFunction(20, null));
IgniteCache<Integer, Integer> cache = ignite.createCache(cfg);
for (int i = 0; i < 100; i++)
cache.put(i, i);
System.out.println("Populated.");
Thread.sleep(2000);
Ignition.stop("server-3", true);
}
private static IgniteConfiguration config(String name) {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setGridName(name);
cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST);
cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends
Event>, int[]>singletonMap(
new IgnitePredicate<Event>() {
@Override public boolean apply(Event event) {
U.debug("EVENT: " + event);
return true;
}
},
new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
));
return cfg;
}
}
-Val
Re: Partition data lost event
Posted by Denis Magda <dm...@apache.org>.
Val, Alex, could anyone of you document this on readme.io <http://readme.io/>?
—
Denis
> On Jan 19, 2017, at 9:49 AM, Valentin Kulichenko <va...@gmail.com> wrote:
>
> Alex,
>
> Thanks for response!
>
> -Val
>
> On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Val et al,
>>
>> Currently partition lost event is fired on nodes which should become
>> partition owners, but did not find other partition owners to rebalance data
>> from. In your example one of the node should have became an owner for
>> partitions 1, 3, the other - for partitions 2, 4 (according to the affinity
>> function assignment), thus this set of events is fired.
>>
>> In Ignite 2.0 lost partitions will be handled on coordinator and events
>> will be fired for all partitions on all cache nodes.
>>
>> --AG
>>
>> 2017-01-18 23:39 GMT+03:00 Denis Magda <dm...@apache.org>:
>>
>>> Alex G. and Yakov should be able to clarify this since they took part in
>>> the creation of the following tickets related to partitions lost
>> consistency
>>> https://issues.apache.org/jira/browse/IGNITE-1605 <
>>> https://issues.apache.org/jira/browse/IGNITE-1605>
>>> https://issues.apache.org/jira/browse/IGNITE-2378 <
>>> https://issues.apache.org/jira/browse/IGNITE-2378>
>>>
>>> —
>>> Denis
>>>
>>>> On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <
>>> valentin.kulichenko@gmail.com> wrote:
>>>>
>>>> Folks,
>>>>
>>>> Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is
>>> fired?
>>>> I just create a small test and the result confused me. This is what I
>> do:
>>>>
>>>> 1. Start several server nodes, all listen to
>>>> EVT_CACHE_REBALANCE_PART_DATA_LOST event.
>>>> 2. Create a cache with no backups.
>>>> 3. Put some values.
>>>> 4. Stop one of the nodes (lose some partitions).
>>>>
>>>> What I noticed is that the event is fired on all nodes that are still
>> in
>>>> topology, but for different partitions. For example, if there are 4
>>>> partitions lost (1,2,3,4), I can get notifications for 1,3 on first
>> node
>>>> and 2,4 on second node. This seems weird and it's not clear to me how
>> to
>>>> use this event.
>>>>
>>>> Can anyone explain this behavior? Is this expected?
>>>>
>>>> Here is the test code:
>>>>
>>>> public class PartitionTest {
>>>> public static void main(String[] args) throws InterruptedException {
>>>> for (int i = 0; i < 4; i++)
>>>> Ignition.start(config("server-" + i));
>>>>
>>>> Ignition.setClientMode(true);
>>>>
>>>> Ignite ignite = Ignition.start();
>>>>
>>>> CacheConfiguration<Integer, Integer> cfg = new
>>>> CacheConfiguration<>("test");
>>>>
>>>> cfg.setAffinity(new RendezvousAffinityFunction(20, null));
>>>>
>>>> IgniteCache<Integer, Integer> cache = ignite.createCache(cfg);
>>>>
>>>> for (int i = 0; i < 100; i++)
>>>> cache.put(i, i);
>>>>
>>>> System.out.println("Populated.");
>>>>
>>>> Thread.sleep(2000);
>>>>
>>>> Ignition.stop("server-3", true);
>>>> }
>>>>
>>>> private static IgniteConfiguration config(String name) {
>>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>>>
>>>> cfg.setGridName(name);
>>>>
>>>> cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_
>> PART_DATA_LOST);
>>>> cfg.setLocalEventListeners(Collections.<IgnitePredicate<?
>> extends
>>>> Event>, int[]>singletonMap(
>>>> new IgnitePredicate<Event>() {
>>>> @Override public boolean apply(Event event) {
>>>> U.debug("EVENT: " + event);
>>>>
>>>> return true;
>>>> }
>>>> },
>>>> new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
>>>> ));
>>>>
>>>> return cfg;
>>>> }
>>>> }
>>>>
>>>> -Val
>>>
>>>
>>
Re: Partition data lost event
Posted by Valentin Kulichenko <va...@gmail.com>.
Alex,
Thanks for response!
-Val
On Wed, Jan 18, 2017 at 10:17 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Val et al,
>
> Currently partition lost event is fired on nodes which should become
> partition owners, but did not find other partition owners to rebalance data
> from. In your example one of the node should have became an owner for
> partitions 1, 3, the other - for partitions 2, 4 (according to the affinity
> function assignment), thus this set of events is fired.
>
> In Ignite 2.0 lost partitions will be handled on coordinator and events
> will be fired for all partitions on all cache nodes.
>
> --AG
>
> 2017-01-18 23:39 GMT+03:00 Denis Magda <dm...@apache.org>:
>
> > Alex G. and Yakov should be able to clarify this since they took part in
> > the creation of the following tickets related to partitions lost
> consistency
> > https://issues.apache.org/jira/browse/IGNITE-1605 <
> > https://issues.apache.org/jira/browse/IGNITE-1605>
> > https://issues.apache.org/jira/browse/IGNITE-2378 <
> > https://issues.apache.org/jira/browse/IGNITE-2378>
> >
> > —
> > Denis
> >
> > > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <
> > valentin.kulichenko@gmail.com> wrote:
> > >
> > > Folks,
> > >
> > > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is
> > fired?
> > > I just create a small test and the result confused me. This is what I
> do:
> > >
> > > 1. Start several server nodes, all listen to
> > > EVT_CACHE_REBALANCE_PART_DATA_LOST event.
> > > 2. Create a cache with no backups.
> > > 3. Put some values.
> > > 4. Stop one of the nodes (lose some partitions).
> > >
> > > What I noticed is that the event is fired on all nodes that are still
> in
> > > topology, but for different partitions. For example, if there are 4
> > > partitions lost (1,2,3,4), I can get notifications for 1,3 on first
> node
> > > and 2,4 on second node. This seems weird and it's not clear to me how
> to
> > > use this event.
> > >
> > > Can anyone explain this behavior? Is this expected?
> > >
> > > Here is the test code:
> > >
> > > public class PartitionTest {
> > > public static void main(String[] args) throws InterruptedException {
> > > for (int i = 0; i < 4; i++)
> > > Ignition.start(config("server-" + i));
> > >
> > > Ignition.setClientMode(true);
> > >
> > > Ignite ignite = Ignition.start();
> > >
> > > CacheConfiguration<Integer, Integer> cfg = new
> > > CacheConfiguration<>("test");
> > >
> > > cfg.setAffinity(new RendezvousAffinityFunction(20, null));
> > >
> > > IgniteCache<Integer, Integer> cache = ignite.createCache(cfg);
> > >
> > > for (int i = 0; i < 100; i++)
> > > cache.put(i, i);
> > >
> > > System.out.println("Populated.");
> > >
> > > Thread.sleep(2000);
> > >
> > > Ignition.stop("server-3", true);
> > > }
> > >
> > > private static IgniteConfiguration config(String name) {
> > > IgniteConfiguration cfg = new IgniteConfiguration();
> > >
> > > cfg.setGridName(name);
> > >
> > > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_
> PART_DATA_LOST);
> > > cfg.setLocalEventListeners(Collections.<IgnitePredicate<?
> extends
> > > Event>, int[]>singletonMap(
> > > new IgnitePredicate<Event>() {
> > > @Override public boolean apply(Event event) {
> > > U.debug("EVENT: " + event);
> > >
> > > return true;
> > > }
> > > },
> > > new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
> > > ));
> > >
> > > return cfg;
> > > }
> > > }
> > >
> > > -Val
> >
> >
>
Re: Partition data lost event
Posted by Alexey Goncharuk <al...@gmail.com>.
Val et al,
Currently partition lost event is fired on nodes which should become
partition owners, but did not find other partition owners to rebalance data
from. In your example one of the node should have became an owner for
partitions 1, 3, the other - for partitions 2, 4 (according to the affinity
function assignment), thus this set of events is fired.
In Ignite 2.0 lost partitions will be handled on coordinator and events
will be fired for all partitions on all cache nodes.
--AG
2017-01-18 23:39 GMT+03:00 Denis Magda <dm...@apache.org>:
> Alex G. and Yakov should be able to clarify this since they took part in
> the creation of the following tickets related to partitions lost consistency
> https://issues.apache.org/jira/browse/IGNITE-1605 <
> https://issues.apache.org/jira/browse/IGNITE-1605>
> https://issues.apache.org/jira/browse/IGNITE-2378 <
> https://issues.apache.org/jira/browse/IGNITE-2378>
>
> —
> Denis
>
> > On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
> >
> > Folks,
> >
> > Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is
> fired?
> > I just create a small test and the result confused me. This is what I do:
> >
> > 1. Start several server nodes, all listen to
> > EVT_CACHE_REBALANCE_PART_DATA_LOST event.
> > 2. Create a cache with no backups.
> > 3. Put some values.
> > 4. Stop one of the nodes (lose some partitions).
> >
> > What I noticed is that the event is fired on all nodes that are still in
> > topology, but for different partitions. For example, if there are 4
> > partitions lost (1,2,3,4), I can get notifications for 1,3 on first node
> > and 2,4 on second node. This seems weird and it's not clear to me how to
> > use this event.
> >
> > Can anyone explain this behavior? Is this expected?
> >
> > Here is the test code:
> >
> > public class PartitionTest {
> > public static void main(String[] args) throws InterruptedException {
> > for (int i = 0; i < 4; i++)
> > Ignition.start(config("server-" + i));
> >
> > Ignition.setClientMode(true);
> >
> > Ignite ignite = Ignition.start();
> >
> > CacheConfiguration<Integer, Integer> cfg = new
> > CacheConfiguration<>("test");
> >
> > cfg.setAffinity(new RendezvousAffinityFunction(20, null));
> >
> > IgniteCache<Integer, Integer> cache = ignite.createCache(cfg);
> >
> > for (int i = 0; i < 100; i++)
> > cache.put(i, i);
> >
> > System.out.println("Populated.");
> >
> > Thread.sleep(2000);
> >
> > Ignition.stop("server-3", true);
> > }
> >
> > private static IgniteConfiguration config(String name) {
> > IgniteConfiguration cfg = new IgniteConfiguration();
> >
> > cfg.setGridName(name);
> >
> > cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST);
> > cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends
> > Event>, int[]>singletonMap(
> > new IgnitePredicate<Event>() {
> > @Override public boolean apply(Event event) {
> > U.debug("EVENT: " + event);
> >
> > return true;
> > }
> > },
> > new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
> > ));
> >
> > return cfg;
> > }
> > }
> >
> > -Val
>
>
Re: Partition data lost event
Posted by Denis Magda <dm...@apache.org>.
Alex G. and Yakov should be able to clarify this since they took part in the creation of the following tickets related to partitions lost consistency
https://issues.apache.org/jira/browse/IGNITE-1605 <https://issues.apache.org/jira/browse/IGNITE-1605>
https://issues.apache.org/jira/browse/IGNITE-2378 <https://issues.apache.org/jira/browse/IGNITE-2378>
—
Denis
> On Jan 18, 2017, at 12:16 PM, Valentin Kulichenko <va...@gmail.com> wrote:
>
> Folks,
>
> Does anyone know how the EVT_CACHE_REBALANCE_PART_DATA_LOST event is fired?
> I just create a small test and the result confused me. This is what I do:
>
> 1. Start several server nodes, all listen to
> EVT_CACHE_REBALANCE_PART_DATA_LOST event.
> 2. Create a cache with no backups.
> 3. Put some values.
> 4. Stop one of the nodes (lose some partitions).
>
> What I noticed is that the event is fired on all nodes that are still in
> topology, but for different partitions. For example, if there are 4
> partitions lost (1,2,3,4), I can get notifications for 1,3 on first node
> and 2,4 on second node. This seems weird and it's not clear to me how to
> use this event.
>
> Can anyone explain this behavior? Is this expected?
>
> Here is the test code:
>
> public class PartitionTest {
> public static void main(String[] args) throws InterruptedException {
> for (int i = 0; i < 4; i++)
> Ignition.start(config("server-" + i));
>
> Ignition.setClientMode(true);
>
> Ignite ignite = Ignition.start();
>
> CacheConfiguration<Integer, Integer> cfg = new
> CacheConfiguration<>("test");
>
> cfg.setAffinity(new RendezvousAffinityFunction(20, null));
>
> IgniteCache<Integer, Integer> cache = ignite.createCache(cfg);
>
> for (int i = 0; i < 100; i++)
> cache.put(i, i);
>
> System.out.println("Populated.");
>
> Thread.sleep(2000);
>
> Ignition.stop("server-3", true);
> }
>
> private static IgniteConfiguration config(String name) {
> IgniteConfiguration cfg = new IgniteConfiguration();
>
> cfg.setGridName(name);
>
> cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST);
> cfg.setLocalEventListeners(Collections.<IgnitePredicate<? extends
> Event>, int[]>singletonMap(
> new IgnitePredicate<Event>() {
> @Override public boolean apply(Event event) {
> U.debug("EVENT: " + event);
>
> return true;
> }
> },
> new int[] {EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST}
> ));
>
> return cfg;
> }
> }
>
> -Val