You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Zhenya Stanilovsky <ar...@mail.ru> on 2021/03/01 06:15:03 UTC
Re[4]: Mixing persistent and in memory cache
hi Stephan, due to logs rebalance still in progress (probably slow network ?) will test pass if you increase sleep interval ? 2 sec fro example ?
Additionally no need to set .setBackups(2) in CacheMode.REPLICATED cache, plz check documentation.
>Hi Zhenya,
>
>your 2nd point: yes, the cache itself has been propagated.
>
>Please be aware that I have successfully used the same test with only the in-memory region as well as with only the persistent region. Only when I combine both, the synchronization stops working for the in memory region.
>
>
>Please find attached the log file (both Ignite nodes run in the same process and contribute to this log file) as well as the current Junit test.
>
>In the log file you will find:
>
>The node startup:
>>>>> starting node A
>>>>> starting node B
>
>The test case startup:
>#### testMem2
>
>The test stops with:
>java.lang.AssertionError: expected:<aval> but was:<null>
> …
> at de.dicos.cpcfe.ignite.IgniteClusterTest.testMem2(IgniteClusterTest.java:175)
>
>---
>Mit freundlichen Grüßen
>
>Stephan Hesse
>Geschäftsführer
>
>DICOS GmbH Kommunikationssysteme
>Alsfelder Straße 11, 64289 Darmstadt
>
>Telefon: +49 6151 82787 27 , Mobil: +49 1761 82787 27
>
>www.dicos.de
>
>DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
>Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
>
>
>
>From: Zhenya Stanilovsky < arzamas123@mail.ru >
>Sent: Friday, February 26, 2021 6:57 AM
>To: user@ignite.apache.org
>Subject: Re[2]: Mixing persistent and in memory cache
>
>
>hi Stephan, something wrong with configuration probably … it`s not expected issue.
>* plz attach somehow or send me ignite.log from all server nodes ?
>* If you change second call :
>IgniteCache<String, String> kva = getInMemoryKeyValue(igA);
>
> IgniteCache<String, String> kvb = getInMemoryKeyValue(igB); ← here
>for something like : IgniteCache<String, String> kvb = getInMemoryKeyValue2(igB);
>
>private IgniteCache<String, String> getInMemoryKeyValue2(Ignite ignite)
> {
> return ignite.cache(new CacheConfiguration<String, String>() <---
>
>just to check that cache has been already created.
>
>Does ignite.cache will see the previously created cache ?
>
>thanks !
>
>
>>Hi Zhenya, thanks for this suggestion.
>>
>>However, neither setting CacheWriteSynchronizationMode to Full Sync nor
>>setting it to FULL_ASYNC changes anything: memory cahce changes do not get
>>propagated:
>>
>>private IgniteCache<String, String> getInMemoryKeyValue(Ignite ignite)
>>{
>>return ignite.getOrCreateCache(new CacheConfiguration<String, String>()
>>.setName("memkv")
>>.setCacheMode(CacheMode.REPLICATED)
>>.setDataRegionName(NodeController.IN_MEMORY_REGION)
>>.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
>>.setBackups(2));
>>}
>>
>>
>>
>>
>>--
>>Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
>
RE: Re[6]: Mixing persistent and in memory cache
Posted by Stephan Hesse <s....@dicos.de>.
Hi Zhenya,
thanks, „baseline“ was the right way to go.
I missed the text in the documentation saying that the baseline is only automatically adjusted for *pure* in memory clusters.
However, I understood that the baseline defines the nodes where information is stored – and does not restrict the nodes from where it can be accessed.
I would have expected that a cluster access on a node not holding the information gets forwarded to a node that has it. This seems not always to be the case(?)
Hint: I had to set the baseline topology explicitly to get it working:
Collection<ClusterNode> nodes = ignite.cluster().forServers().nodes();
ignite.cluster().setBaselineTopology(nodes);
or:
ignite.cluster().baselineAutoAdjustEnabled(true);
ignite.cluster().baselineAutoAdjustTimeout(30000);
---
Mit freundlichen Grüßen
Stephan Hesse
Geschäftsführer
DICOS GmbH Kommunikationssysteme
Alsfelder Straße 11, 64289 Darmstadt
Telefon: +49 6151 82787 27, Mobil: +49 1761 82787 27
www.dicos.de<http://www.dicos.de/>
DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
From: Zhenya Stanilovsky <ar...@mail.ru>
Sent: Tuesday, March 2, 2021 7:03 AM
To: user@ignite.apache.org
Subject: Re[6]: Mixing persistent and in memory cache
Ok i found !
18:36:07 noringBase.info INFO Topology snapshot [ver=2, locNode=2bf85583, servers=2, clients=0, state=ACTIVE, CPUs=8, offheap=6.3GB, heap=3.5GB]
18:36:07 noringBase.info INFO ^-- Baseline [id=0, size=1, online=1, offline=0]
you call baseline command after first node was started, thus you have 2 alive nodes with only one in baseline.
clear your persistent directories, and rewrite code like :
startNode(...a..)
ignite = startNode(..b...)
ignite.cluster().state(ClusterState.ACTIVE);
Hi Zhenya,
To be on the safe side, I increased the sleep to 5 sec. I also removed the setBackups(2) – no difference – the test is still failing!
Newest log file with corresponding source attached.
I added more specific logs:
18:36:07 lusterTest.testMem2 INFO >>>>>> value has been written to 'a': aval
18:36:12 lusterTest.testMem2 INFO >>>>>> value retrieved from 'b': null
---
Mit freundlichen Grüßen
Stephan Hesse
Geschäftsführer
DICOS GmbH Kommunikationssysteme
Alsfelder Straße 11, 64289 Darmstadt
Telefon: +49 6151 82787 27, Mobil: +49 1761 82787 27
www.dicos.de<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dicos.de%2F&data=04%7C01%7Cs.hesse%40dicos.de%7Cd83cdcaac247472db9f208d8dd40de9a%7Cac9cfe7f98014e47b9fef4a671e0ab36%7C1%7C0%7C637502618570355193%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FL%2B%2F6kJfF40FnajOpX6uWQY4WuvOVuO5ImLT5FPGAcc%3D&reserved=0>
DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
From: Zhenya Stanilovsky <ar...@mail.ru>>
Sent: Monday, March 1, 2021 7:15 AM
To: user@ignite.apache.org</c...@ignite.apache.org>
Subject: Re[4]: Mixing persistent and in memory cache
hi Stephan, due to logs rebalance still in progress (probably slow network ?) will test pass if you increase sleep interval ? 2 sec fro example ?
Additionally no need to set .setBackups(2) in CacheMode.REPLICATED cache, plz check documentation.
Hi Zhenya,
your 2nd point: yes, the cache itself has been propagated.
Please be aware that I have successfully used the same test with only the in-memory region as well as with only the persistent region. Only when I combine both, the synchronization stops working for the in memory region.
Please find attached the log file (both Ignite nodes run in the same process and contribute to this log file) as well as the current Junit test.
In the log file you will find:
The node startup:
>>>> starting node A
>>>> starting node B
The test case startup:
#### testMem2
The test stops with:
java.lang.AssertionError: expected:<aval> but was:<null>
…
at de.dicos.cpcfe.ignite.IgniteClusterTest.testMem2(IgniteClusterTest.java:175)
---
Mit freundlichen Grüßen
Stephan Hesse
Geschäftsführer
DICOS GmbH Kommunikationssysteme
Alsfelder Straße 11, 64289 Darmstadt
Telefon: +49 6151 82787 27, Mobil: +49 1761 82787 27
www.dicos.de<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dicos.de%2F&data=04%7C01%7Cs.hesse%40dicos.de%7Cd83cdcaac247472db9f208d8dd40de9a%7Cac9cfe7f98014e47b9fef4a671e0ab36%7C1%7C0%7C637502618570355193%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FL%2B%2F6kJfF40FnajOpX6uWQY4WuvOVuO5ImLT5FPGAcc%3D&reserved=0>
DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
From: Zhenya Stanilovsky <ar...@mail.ru>>
Sent: Friday, February 26, 2021 6:57 AM
To: user@ignite.apache.org<ma...@ignite.apache.org>
Subject: Re[2]: Mixing persistent and in memory cache
hi Stephan, something wrong with configuration probably … it`s not expected issue.
1. plz attach somehow or send me ignite.log from all server nodes ?
2. If you change second call :
IgniteCache<String, String> kva = getInMemoryKeyValue(igA);
IgniteCache<String, String> kvb = getInMemoryKeyValue(igB); ← here
for something like : IgniteCache<String, String> kvb = getInMemoryKeyValue2(igB);
private IgniteCache<String, String> getInMemoryKeyValue2(Ignite ignite)
{
return ignite.cache(new CacheConfiguration<String, String>() <---
just to check that cache has been already created.
Does ignite.cache will see the previously created cache ?
thanks !
Hi Zhenya, thanks for this suggestion.
However, neither setting CacheWriteSynchronizationMode to Full Sync nor
setting it to FULL_ASYNC changes anything: memory cahce changes do not get
propagated:
private IgniteCache<String, String> getInMemoryKeyValue(Ignite ignite)
{
return ignite.getOrCreateCache(new CacheConfiguration<String, String>()
.setName("memkv")
.setCacheMode(CacheMode.REPLICATED)
.setDataRegionName(NodeController.IN_MEMORY_REGION)
.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
.setBackups(2));
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=04%7C01%7Cs.hesse%40dicos.de%7Cd83cdcaac247472db9f208d8dd40de9a%7Cac9cfe7f98014e47b9fef4a671e0ab36%7C1%7C0%7C637502618570365190%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pUh1fmpvEIoFg79lRx%2F%2FEsOAff6JZA5XZJVcUZi75wU%3D&reserved=0>
Re[6]: Mixing persistent and in memory cache
Posted by Zhenya Stanilovsky <ar...@mail.ru>.
Ok i found !
18:36:07 noringBase.info INFO Topology snapshot [ver=2, locNode=2bf85583, servers=2, clients=0, state=ACTIVE, CPUs=8, offheap=6.3GB, heap=3.5GB]
18:36:07 noringBase.info INFO ^-- Baseline [id=0, size=1, online=1, offline=0]
you call baseline command after first node was started, thus you have 2 alive nodes with only one in baseline.
clear your persistent directories, and rewrite code like :
startNode(...a..)
ignite = startNode(..b...)
ignite.cluster().state(ClusterState.ACTIVE);
>
>>
>>>Hi Zhenya,
>>>To be on the safe side, I increased the sleep to 5 sec. I also removed the setBackups(2) – no difference – the test is still failing!
>>>
>>>Newest log file with corresponding source attached.
>>>
>>>I added more specific logs:
>>>
>>>18:36:07 lusterTest.testMem2 INFO >>>>>> value has been written to 'a': aval
>>>18:36:12 lusterTest.testMem2 INFO >>>>>> value retrieved from 'b': null
>>>
>>>---
>>>Mit freundlichen Grüßen
>>>
>>>Stephan Hesse
>>>Geschäftsführer
>>>
>>>DICOS GmbH Kommunikationssysteme
>>>Alsfelder Straße 11, 64289 Darmstadt
>>>
>>>Telefon: +49 6151 82787 27 , Mobil: +49 1761 82787 27
>>>
>>>www.dicos.de
>>>
>>>DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
>>>Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
>>>
>>>
>>>
>>>From: Zhenya Stanilovsky < arzamas123@mail.ru >
>>>Sent: Monday, March 1, 2021 7:15 AM
>>>To: user@ignite.apache.org
>>>Subject: Re[4]: Mixing persistent and in memory cache
>>>
>>>
>>>hi Stephan, due to logs rebalance still in progress (probably slow network ?) will test pass if you increase sleep interval ? 2 sec fro example ?
>>>Additionally no need to set .setBackups(2) in CacheMode.REPLICATED cache, plz check documentation.
>>>
>>>
>>>>Hi Zhenya,
>>>>
>>>>your 2nd point: yes, the cache itself has been propagated.
>>>>
>>>>Please be aware that I have successfully used the same test with only the in-memory region as well as with only the persistent region. Only when I combine both, the synchronization stops working for the in memory region.
>>>>
>>>>
>>>>Please find attached the log file (both Ignite nodes run in the same process and contribute to this log file) as well as the current Junit test.
>>>>
>>>>In the log file you will find:
>>>>
>>>>The node startup:
>>>>>>>> starting node A
>>>>>>>> starting node B
>>>>
>>>>The test case startup:
>>>>#### testMem2
>>>>
>>>>The test stops with:
>>>>java.lang.AssertionError: expected:<aval> but was:<null>
>>>> …
>>>> at de.dicos.cpcfe.ignite.IgniteClusterTest.testMem2(IgniteClusterTest.java:175)
>>>>
>>>>---
>>>>Mit freundlichen Grüßen
>>>>
>>>>Stephan Hesse
>>>>Geschäftsführer
>>>>
>>>>DICOS GmbH Kommunikationssysteme
>>>>Alsfelder Straße 11, 64289 Darmstadt
>>>>
>>>>Telefon: +49 6151 82787 27 , Mobil: +49 1761 82787 27
>>>>
>>>>www.dicos.de
>>>>
>>>>DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
>>>>Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
>>>>
>>>>
>>>>
>>>>From: Zhenya Stanilovsky < arzamas123@mail.ru >
>>>>Sent: Friday, February 26, 2021 6:57 AM
>>>>To: user@ignite.apache.org
>>>>Subject: Re[2]: Mixing persistent and in memory cache
>>>>
>>>>
>>>>hi Stephan, something wrong with configuration probably … it`s not expected issue.
>>>>* plz attach somehow or send me ignite.log from all server nodes ?
>>>>* If you change second call :
>>>>IgniteCache<String, String> kva = getInMemoryKeyValue(igA);
>>>>
>>>> IgniteCache<String, String> kvb = getInMemoryKeyValue(igB); ← here
>>>>for something like : IgniteCache<String, String> kvb = getInMemoryKeyValue2(igB);
>>>>
>>>>private IgniteCache<String, String> getInMemoryKeyValue2(Ignite ignite)
>>>> {
>>>> return ignite.cache(new CacheConfiguration<String, String>() <---
>>>>
>>>>just to check that cache has been already created.
>>>>
>>>>Does ignite.cache will see the previously created cache ?
>>>>
>>>>thanks !
>>>>
>>>>
>>>>>Hi Zhenya, thanks for this suggestion.
>>>>>
>>>>>However, neither setting CacheWriteSynchronizationMode to Full Sync nor
>>>>>setting it to FULL_ASYNC changes anything: memory cahce changes do not get
>>>>>propagated:
>>>>>
>>>>>private IgniteCache<String, String> getInMemoryKeyValue(Ignite ignite)
>>>>>{
>>>>>return ignite.getOrCreateCache(new CacheConfiguration<String, String>()
>>>>>.setName("memkv")
>>>>>.setCacheMode(CacheMode.REPLICATED)
>>>>>.setDataRegionName(NodeController.IN_MEMORY_REGION)
>>>>>.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
>>>>>.setBackups(2));
>>>>>}
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>--
>>>>>Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
RE: Re[4]: Mixing persistent and in memory cache
Posted by Stephan Hesse <s....@dicos.de>.
Hi Zhenya,
To be on the safe side, I increased the sleep to 5 sec. I also removed the setBackups(2) – no difference – the test is still failing!
Newest log file with corresponding source attached.
I added more specific logs:
18:36:07 lusterTest.testMem2 INFO >>>>>> value has been written to 'a': aval
18:36:12 lusterTest.testMem2 INFO >>>>>> value retrieved from 'b': null
---
Mit freundlichen Grüßen
Stephan Hesse
Geschäftsführer
DICOS GmbH Kommunikationssysteme
Alsfelder Straße 11, 64289 Darmstadt
Telefon: +49 6151 82787 27, Mobil: +49 1761 82787 27
www.dicos.de<http://www.dicos.de/>
DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
From: Zhenya Stanilovsky <ar...@mail.ru>
Sent: Monday, March 1, 2021 7:15 AM
To: user@ignite.apache.org
Subject: Re[4]: Mixing persistent and in memory cache
hi Stephan, due to logs rebalance still in progress (probably slow network ?) will test pass if you increase sleep interval ? 2 sec fro example ?
Additionally no need to set .setBackups(2) in CacheMode.REPLICATED cache, plz check documentation.
Hi Zhenya,
your 2nd point: yes, the cache itself has been propagated.
Please be aware that I have successfully used the same test with only the in-memory region as well as with only the persistent region. Only when I combine both, the synchronization stops working for the in memory region.
Please find attached the log file (both Ignite nodes run in the same process and contribute to this log file) as well as the current Junit test.
In the log file you will find:
The node startup:
>>>> starting node A
>>>> starting node B
The test case startup:
#### testMem2
The test stops with:
java.lang.AssertionError: expected:<aval> but was:<null>
…
at de.dicos.cpcfe.ignite.IgniteClusterTest.testMem2(IgniteClusterTest.java:175)
---
Mit freundlichen Grüßen
Stephan Hesse
Geschäftsführer
DICOS GmbH Kommunikationssysteme
Alsfelder Straße 11, 64289 Darmstadt
Telefon: +49 6151 82787 27, Mobil: +49 1761 82787 27
www.dicos.de<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dicos.de%2F&data=04%7C01%7Cs.hesse%40dicos.de%7Ceb33bfca360f4995049108d8dc796429%7Cac9cfe7f98014e47b9fef4a671e0ab36%7C1%7C0%7C637501761802741015%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WFyGQGOQFBUndUFTi%2BoKaYgqMp6h5CP3qOkbL0GR0MU%3D&reserved=0>
DICOS GmbH Kommunikationssysteme, Darmstadt, Amtsgericht Darmstadt HRB 7024,
Geschäftsführer: Dr. Winfried Geyer, Stephan Hesse, Waldemar Wiesner
From: Zhenya Stanilovsky <ar...@mail.ru>>
Sent: Friday, February 26, 2021 6:57 AM
To: user@ignite.apache.org</c...@ignite.apache.org>
Subject: Re[2]: Mixing persistent and in memory cache
hi Stephan, something wrong with configuration probably … it`s not expected issue.
1. plz attach somehow or send me ignite.log from all server nodes ?
2. If you change second call :
IgniteCache<String, String> kva = getInMemoryKeyValue(igA);
IgniteCache<String, String> kvb = getInMemoryKeyValue(igB); ← here
for something like : IgniteCache<String, String> kvb = getInMemoryKeyValue2(igB);
private IgniteCache<String, String> getInMemoryKeyValue2(Ignite ignite)
{
return ignite.cache(new CacheConfiguration<String, String>() <---
just to check that cache has been already created.
Does ignite.cache will see the previously created cache ?
thanks !
Hi Zhenya, thanks for this suggestion.
However, neither setting CacheWriteSynchronizationMode to Full Sync nor
setting it to FULL_ASYNC changes anything: memory cahce changes do not get
propagated:
private IgniteCache<String, String> getInMemoryKeyValue(Ignite ignite)
{
return ignite.getOrCreateCache(new CacheConfiguration<String, String>()
.setName("memkv")
.setCacheMode(CacheMode.REPLICATED)
.setDataRegionName(NodeController.IN_MEMORY_REGION)
.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
.setBackups(2));
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/<https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=04%7C01%7Cs.hesse%40dicos.de%7Ceb33bfca360f4995049108d8dc796429%7Cac9cfe7f98014e47b9fef4a671e0ab36%7C1%7C0%7C637501761802751011%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4OkApZIjwBBnTp6y4cwVDUqsuH6OBwz4qWAWcn6da4Y%3D&reserved=0>